This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Inconsistent behavior with slice assignment?
Type: behavior Stage:
Components: Documentation, Interpreter Core Versions: Python 3.6, Python 3.4, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Massimiliano Culpo, docs@python, r.david.murray, serhiy.storchaka, skrah, steven.daprano
Priority: normal Keywords:

Created on 2017-12-12 11:58 by Massimiliano Culpo, last changed 2022-04-11 14:58 by admin.

Messages (10)
msg308120 - (view) Author: Massimiliano Culpo (Massimiliano Culpo) Date: 2017-12-12 11:58
I am not really sure if what I am reporting is to be considered a bug in the interpreter (slice built-in), a lack in documentation or if it is just me not finding the proper references (in which case feel free to close this and sorry for the noise).

Anyhow, while trying to code a class that mimics built-in lists, I noticed a few seemingly inconsistent behaviors in the use of slices to set items.

This can be reduced down to this very simple example:
```
Python 3.6.2 (default, Jul 29 2017, 00:00:00) 
[GCC 4.8.4] on linux

# 1. Slice has start and stop, step is defaulted to None
>>> a = [1, 2, 3, 4]
>>> a[1:3] = [11, 22, 33, 44]
>>> a
[1, 11, 22, 33, 44, 4]

# 2. Same as above setting explicitly step == 1
>>> a = [1, 2, 3, 4]
>>> a[1:3:1] = [11, 22, 33, 44]
>>> a
[1, 11, 22, 33, 44, 4]

# 3. Trying to do the same thing with step == -1
# (I naively expected either [1, 44, 33, 22, 11, 4] or 2. to fail like 3.)
>>> a = [1, 2, 3, 4]
>>> a[3:1:-1] = [11, 22, 33, 44]
Traceback (most recent call last):
  File "<input>", line 1, in <module>
ValueError: attempt to assign sequence of size 4 to extended slice of size 2
```

Now, the first issue: I tried to search what an "extended slice" is exactly, but found nothing in the docs. What is written here

https://docs.python.org/3/library/functions.html#slice

seems to suggest that an "extended slice" is a slice where `step is not None`, but then I would expect case 2. to behave the same as case 3. So to me it appears as either lack of documentation on the behavior, or a bug in 2.

Second issue: in the same page in the docs it's written

> slice objects are also generated when extended indexing syntax is used. For example: a[start:stop:step] or a[start:stop, i]

I wasn't aware of the second syntax, and tried to use it:
```
>>> a = [1, 2, 3, 4]
>>> a[1:3:1]
[2, 3]
>>> a[1:3, 1]
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: list indices must be integers or slices, not tuple
```
I guess that's not the result one would expect...

Third issue: slightly related, but here

https://docs.python.org/3/library/stdtypes.html#mutable-sequence-types

I see that the documentation of `insert` reads:

> inserts x into s at the index given by i (same as s[i:i] = [x])

and from the interpreter:
```
help(list.insert)
Help on method_descriptor:
insert(...)
    L.insert(index, object) -- insert object before index

```

Here the fact is that `insert` really behaves like `s[i:i] = [x]`, but in my understanding it doesn't keep the promise to `insert object before index`.
```
# "Out of bounds" on the right
>>> a = [1, 2, 3, 4]
>>> a.insert(101, 11)
>>> a
[1, 2, 3, 4, 11]

# "Out of bounds" on the left
>>> a.insert(-101, 11)
>>> a
[11, 1, 2, 3, 4]
```
I don't know honestly if docs are not clear here, or if it is just me, but I didn't expect this behavior and thought it was worth reporting.

Let me know if you need more information.
msg308123 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2017-12-12 12:39
Please don't report three issues under one ticket, unless they are so closely related that they cannot be separated.

I don't understand what your second issue actually is. You refer to the docs that specify extended syntax as [start:stop:step] but then you tried [start:stop,step] which is correctly an error. (Notice the comma instead of colon.)

And again, your third issue to do with list.insert ... what is your actual bug report? Have you found an example where these aren't equivalent?

alist.insert(pos, value)

alist[pos:pos] = [value]

I don't understand your third bug report here... you show two examples where the insert function works as documented. You say you didn't expect the behaviour, but what behaviour did you expect?
msg308124 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-12-12 12:46
Python lists don't implement extended slices, try numpy arrays:

>>> import numpy as np
>>> x = array([[1,2,3], [4,5,6]])
>>> x[:, 2]
array([3, 6])
>>> 
>>> lst = [[1,2,3], [4,5,6]]
>>> lst[:, 2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: list indices must be integers or slices, not tuple
>>> 

Was there any issue apart from the extended slices?
msg308126 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-12-12 13:05
I see the first issue now and I agree that Python behaves strangely.

Numpy
=====

>>> x = array([1,2,3])
>>> x[1:2] = [1,2,3,4,5]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: cannot copy sequence with size 5 to array axis with dimension 1


Python
======

>>> lst = [1,2,3]
>>> lst[1:2] = [1,2,3,4,5]
>>> lst
[1, 1, 2, 3, 4, 5, 3]
msg308129 - (view) Author: Massimiliano Culpo (Massimiliano Culpo) Date: 2017-12-12 13:37
Steven and Stefan, thanks for the quick replies, even though they are giving slightly different answers to the same questions :-)

@steven.daprano
> You refer to the docs that specify extended syntax as [start:stop:step] but then you tried [start:stop,step] which is correctly an error. (Notice the comma instead of colon.)

Maybe you missed the quote from the docs: "For example: a[start:stop:step] or a[start:stop, i]"

You say that the docs "specify extended syntax as [start:stop:step]". Apparently you're also surprised this syntax exists, and didn't notice that the error is not a SyntaxError, but a TypeError. I argue then that the syntax is there. If I get correctly what @skrah says, he seems to suggest that the syntax is there, but no built-in type is using it. Am I right?

> And again, your third issue to do with list.insert ... what is your actual bug report? Have you found an example where these aren't equivalent?

I didn't say that. I actually said "`insert` really behaves like `s[i:i] = [x]`". But the docs from the interpreter say:
```
help(list.insert)
Help on method_descriptor:
insert(...)
    L.insert(index, object) -- insert object before index

```
What does it mean to insert an object before index 101 or -101 of a 4 items list? I think if the help said "-- same as L[index:index] = [object]" I would have been fine with it.

> Please don't report three issues under one ticket, unless they are so closely related that they cannot be separated.

Apologies if this was not correct (I see you started an issue with a smaller scope). But I thought they were closely related, because all three might have been fixed by a rewording of a few parts of the docs dealing with slices.

@skrah
> I see the first issue now and I agree that Python behaves strangely.

To be clear, I don't have issues with:
```
>>> lst = [1,2,3]
>>> lst[1:2] = [1,2,3,4,5]
>>> lst
[1, 1, 2, 3, 4, 5, 3]
```
in Python, which probably is a widely used idiom to modify lists. 

My concerns are with the fact that you get the same behavior when you use `lst[1:2:1]` (which in my understanding is an extended slice), but you get a `ValueError` for any other value of `slice.step`. That might be something worth clarifying as part of issue32289 maybe?
msg308130 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-12-12 13:47
Okay, the different meanings of "extended slicing" seem to be an old
problem:

https://mail.python.org/pipermail/tutor/2007-July/055838.html


I assumed it referred to numpy's tuple indexing syntax, but it is
apparently also used for referring to the step value of a single
slice object.


Somehow I've always used the term for the tuple syntax.
msg308131 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-12-12 14:10
It actually makes sense that a slice assignment with a different length replacement list with a step of 1 works but any other step doesn't.  Logically you can see that you can cut out a chunk and replace it with a different sized chunk, but you can't do that if the step is not abs(1).  So a doc tweak may be appropriate.  And it is surprising that -1 doesn't work (but why would you ever use it? :), so that might be a buglet.

For a numpy array, I imagine the rules are different, so an error there on stride 1 is not that surprising either, though *perhaps* there is room for improvement on the numpy side.  (In other words, the stride 1 python behavior is "correct" from a python language perspective.)

And yes, it would be much easier to discuss these separate issues in separate issues.  Now you know for next time :)
msg308133 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-12-12 14:34
Thanks, David, you are of course right! -- I've worked so much with fixed arrays in the past year that Python's behavior seemed odd and unfathomable. :-)


[Numpy arrays can't adopt that behavior, because growing them would be too expensive.]
msg316319 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-05-09 09:33
So this issue can be closed now?
msg316347 - (view) Author: Massimiliano Culpo (Massimiliano Culpo) Date: 2018-05-10 05:16
Hi Serhiy,

I briefly checked out Python 3.7.0b3, and saw no changes with respect to:

   - Documenting what is the exact meaning of "extended slice" (but that
   now is also tracked by issue 32289)
   - Improving the docstring of list.insert (to say that [1, 2, 3,
   4].insert(-101, 0) is allowed?)
   - Fixing the inconsistent behavior for slices with step == -1

If I missed anything, please let me know. Otherwise, from my side, I think
it only depends on whether you want to track these issues (or even if you
consider any of the above an issue).

Cheers,

On Wed, May 9, 2018 at 11:33 AM, Serhiy Storchaka <report@bugs.python.org>
wrote:

>
> Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment:
>
> So this issue can be closed now?
>
> ----------
> nosy: +serhiy.storchaka
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue32288>
> _______________________________________
>
History
Date User Action Args
2022-04-11 14:58:55adminsetgithub: 76469
2018-05-10 05:16:28Massimiliano Culposetmessages: + msg316347
2018-05-09 09:33:06serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg316319
2017-12-12 14:34:22skrahsetmessages: + msg308133
2017-12-12 14:10:32r.david.murraysetnosy: + r.david.murray
messages: + msg308131
2017-12-12 13:47:31skrahsetmessages: + msg308130
2017-12-12 13:37:57Massimiliano Culposetmessages: + msg308129
2017-12-12 13:05:25skrahsetmessages: + msg308126
2017-12-12 12:46:19skrahsetnosy: + skrah
messages: + msg308124
2017-12-12 12:39:47steven.dapranosetnosy: + steven.daprano
messages: + msg308123
2017-12-12 11:58:23Massimiliano Culpocreate