This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Make itertools.islice supports negative values for start and stop arguments for sized iterable object
Type: enhancement Stage: resolved
Components: Library (Lib) Versions:
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: rhettinger, serhiy.storchaka, tttnns, ynikitenko
Priority: normal Keywords: patch

Created on 2018-03-10 09:18 by tttnns, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
islice.patch tttnns, 2018-03-10 09:18
Messages (6)
msg313517 - (view) Author: TitanSnow (tttnns) * Date: 2018-03-10 09:18
``islice()`` does not support negative values for start or stop,
which does not matter for plain iterators.
However, for some cases, we have a sized iterable object
which is not subscriptable, using ``islice()`` makes
code ugly::

    d = OrderedDict()
    for i in range(10): d[i] = i
    dv = d.keys()
    # dv is a KeysView which is a sized iterable

    # now I wanna get a slice of dv which does not contain the last element
    islice(dv, len(dv) - 1)

As it shows, I have to use ``len(dv)`` to get its length.

For sized iterable objects, ``islice()`` could
support negative values for start or stop.
In this way, the above code can be written like this::

    islice(dv, -1)

For iterable objects which is not sized,
it could still be not supported::

    islice(iter(range(10)), -1)

raises a ValueError as its original behavior.
msg313519 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-03-10 10:30
I do not think this is suitable for the itertools module. It is designed to work with iterators, i.e. with objects that support the iterator protocol. What if the iterable change the size during iteration? Should the islice iterator produce less or larger number of items than the initial estimation? Or detect this change and raise an error? Should it revive the iteration if new items were added after consuming the last item? These design questions should have thoughtful answers. And handling all corner cases will complicate the code. I afraid that after adding support of negative start and stop we will get a request for supporting negative step.

It is possible to implement support of negative indices for arbitrary iterators using tee() or deque, but this will complicate the islice() code even more for a small niche case.

This may be a use case for the new module that works specially with sequences. It will be added once we will have enough use cases.
msg313545 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-03-10 16:58
I concur with Serhiy.  This doesn't make sense for itertools.

Marking this as closed. Thank you for the suggestion.
msg313636 - (view) Author: TitanSnow (tttnns) * Date: 2018-03-12 09:20
Now I have thought about it and realized that
it's not suitable for islice.
But there's still a common use case to
drop some elements from the beginning or
ending of a iterable, which is also a main
reason why I wanted islice to support
negative values of start and stop.
So how about adding two functions
``drop_first`` and ``drop_last``
to do such things. They can be implemented
like this, in which way they support
arbitrary iterators::

    def drop_first(iterable, n=1):
        for _ in range(n):
            next(iterable)
        for e in iterable:
            yield e

    def drop_last(iterable, n=1):
        dq = deque()
        for _ in range(n):
            dq.append(next(iterable))
        for e in iterable:
            dq.append(e)
            yield dq.popleft()
msg392246 - (view) Author: Yaroslav Nikitenko (ynikitenko) Date: 2021-04-28 19:02
I hope it's fine to add to closed topics here.

I agree with the decision that islice should not handle a special case of sized containers vs iterables.

However, I think that the support of negative indices in islice would be nice. A simple use case would be to get the last element of an iterable.
I had to write my own code for that (you may read and/or use it if needed), because I needed this algorithm in my analysis. Here is the link to my function ISlice._run_negative_islice in my architectural framework for data analysis: https://github.com/ynikitenko/lena/blob/master/lena/flow/iterators.py#L150 or on https://lena.readthedocs.io/en/latest/flow.html#lena.flow.iterators.ISlice (I'm afraid the correct link might change later).

I also agree that to support negative indices would require more work. Indeed, it seems that for each combination of positive/negative start/stop it required a new algorithm! I didn't implement negative steps.

However, I think that because it's not easy to implement, it would be even better to include that in the standard library (because it would save other people from writing that).

If we care about code reuse: I think that negative indices make the algorithm more useful, while non-trivial steps are redundant, because they can be implemented by a composition of a slice with step=1 with a simple slice with start, stop=None, None with step!=1.

Negative slice fundamentally changes the algorithm: if one wants to have the flow inverted, fflow[0, None, -1], one would have to store it all in memory! Anyway it's easy to make this in a separate function. So I think it's more functional to implement negative indices and discard negative steps (or non-trivial steps at all).

About drop_first, drop_last suggested above: I also think that they are redundant and would be better incorporated into islice.
msg392249 - (view) Author: Yaroslav Nikitenko (ynikitenko) Date: 2021-04-28 19:16
Sorry for a typo. The paragraph before the last should read

Negative *step* fundamentally changes the algorithm:... flow[-1:None:-1].
History
Date User Action Args
2022-04-11 14:58:58adminsetgithub: 77221
2021-04-28 19:16:14ynikitenkosetmessages: + msg392249
2021-04-28 19:02:53ynikitenkosetnosy: + ynikitenko
messages: + msg392246
2018-03-12 09:20:43tttnnssetmessages: + msg313636
2018-03-10 16:58:30rhettingersetstatus: open -> closed
resolution: rejected
messages: + msg313545

stage: resolved
2018-03-10 10:30:36serhiy.storchakasetnosy: + rhettinger, serhiy.storchaka
messages: + msg313519
2018-03-10 09:18:46tttnnscreate