classification
Title: inconsistent result when concatenating list with iterators
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.2
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: belopolsky Nosy List: amaury.forgeotdarc, belopolsky, djc, eric.araujo, ysj.ray
Priority: normal Keywords:

Created on 2010-07-20 11:28 by djc, last changed 2010-07-21 18:10 by belopolsky. This issue is now closed.

Messages (15)
msg110887 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2010-07-20 11:28
This seems wrong:

>>> a = []
>>> b = iter(['c', 'd'])
>>> a += b
>>> c = []
>>> c + iter(['d', 'e'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can only concatenate list (not "listiterator") to list

In other words, if iadd can deal with it, why can't add?
msg110937 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-07-20 18:04
This actually works for any iterator:
>>> l = []
>>> l += 'abc'
>>> l
['a', 'b', 'c']
msg110939 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-07-20 18:04
for any *iterable*!
msg110948 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-20 19:00
> In other words, if iadd can deal with it, why can't add?

Should ['a', 'b', 'c'] + 'def' return ['a, 'b', 'c', 'd' , 'e', 'f'] or 'abcdef'?

"In the face of ambiguity, refuse the temptation to guess."
msg110950 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2010-07-20 19:04
Sure, but:

>>> a = ['a', 'b', 'c']
>>> a += 'def'
>>> a
['a', 'b', 'c', 'd', 'e', 'f']

I think either way is fine (I'm probably on the side of refusing to guess), but what I don't understand is why a += b is different from a + b in this respect.
msg110953 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-20 19:07
> I don't understand is why a += b is different from a + b
> in this respect.

When a is mutable, a += b updates it in-place, so there is no ambiguity: the type of a cannot change.  When you do a + b, there is no reason to treat a as more deserving than b when selecting the type of the result.
msg110954 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-20 19:10
Note that immutable types are consistent:

>>> x = tuple('abc')
>>> x += 'def'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can only concatenate tuple (not "str") to tuple

>>> x + 'def'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can only concatenate tuple (not "str") to tuple
msg110955 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2010-07-20 19:14
Hmm, that's pretty subtle. I guess I understand, at least.
msg110959 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-07-20 19:53
Alexander, why the pending status?

Dirkjan, do you think this is a doc bug?
msg110960 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-20 20:09
> why the pending status?

To me this is a way to say "I will close this next time I look unless someone will change my mind."  "pending" has a nice property that it will change to "open" once someone adds a comment.  This is exactly what I wanted in this case.

> do you think this is a doc bug?

Interestingly, I could not find any discussion of += for lists in the docs.  This maybe because "+=" is not really search friendly.  In any case, a doc bug (if any) should be reported under a separate issue.
msg110968 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-07-20 20:33
Thanks for the explanation of pending, I thought the mysterious Roundup/browser bug had changed the status in my back.

Searching for __iadd__ gave me only library/collections; library/stdtypes says very little.

I thought it was common to requalify a bug report into a doc bug.
msg110970 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-20 20:37
> I thought it was common to requalify a bug report into a doc bug.

That's upto you and OP.  My rule of thumb is if the title reads as either code or doc bug, it's ok to reclassify.  If not, it is better to open a separate issue.
msg110971 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2010-07-20 20:49
Hmm, I'm still not entirely sure that += shouldn't disallow this as well, just because of the inconsistency. I understand the reasoning now, but it feels like maybe it should be an implementation detail, and it should just be disallowed (refuse to guess).
msg111008 - (view) Author: ysj.ray (ysj.ray) Date: 2010-07-21 06:09
Alexander:

> When a is mutable, a += b updates it in-place, so there is no ambiguity: the type of a cannot change.  When you do a + b, there is no reason to treat a as more deserving than b when selecting the type of the result

Does this means "a += b" is not the same as "a = a + b"? I'd think this can be seen as a pitfall for python.
msg111069 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-21 14:35
On Wed, Jul 21, 2010 at 2:09 AM, Ray.Allen <report@bugs.python.org> wrote:
..
> Does this means "a += b" is not the same as "a = a + b"?

For immutable a, the two are practically the same, for mutable, they
are necessarily different.   This is explained in __iadd__
documentation:

"""
These methods are called to implement the augmented arithmetic
assignments (+=, -=, *=, /=, //=, %=, **=, <<=, >>=, &=, ^=, |=).
These methods should attempt to do the operation in-place (modifying
self) and return the result (which could be, but does not have to be,
self). If a specific method is not defined, the augmented assignment
falls back to the normal methods. For instance, to execute the
statement x += y, where x is an instance of a class that has an
__iadd__() method, x.__iadd__(y) is called. If x is an instance of a
class that does not define a __iadd__() method, x.__add__(y) and
y.__radd__(x) are considered, as with the evaluation of x + y.
"""  http://docs.python.org/reference/datamodel.html?#object.__iadd__

> I'd think this can be seen as a pitfall for python.

No.  Please see PEP 203 for the rationale.
http://www.python.org/dev/peps/pep-0203/
History
Date User Action Args
2010-07-21 18:10:47belopolskysetstatus: open -> closed
2010-07-21 14:35:19belopolskysetmessages: + msg111069
2010-07-21 06:09:36ysj.raysetnosy: + ysj.ray
messages: + msg111008
2010-07-20 20:49:32djcsetmessages: + msg110971
2010-07-20 20:37:51belopolskysetmessages: + msg110970
2010-07-20 20:33:59eric.araujosetmessages: + msg110968
2010-07-20 20:09:53belopolskysetstatus: pending -> open

messages: + msg110960
2010-07-20 19:54:40eric.araujosetstatus: open -> pending
2010-07-20 19:53:53eric.araujosetstatus: pending -> open

messages: + msg110959
2010-07-20 19:15:43belopolskysetstatus: open -> pending
assignee: belopolsky
resolution: rejected
stage: resolved
2010-07-20 19:14:18djcsetmessages: + msg110955
2010-07-20 19:10:59belopolskysetmessages: + msg110954
2010-07-20 19:08:39belopolskysettype: enhancement
versions: + Python 3.2, - Python 2.6
2010-07-20 19:07:30belopolskysetmessages: + msg110953
2010-07-20 19:04:12djcsetmessages: + msg110950
2010-07-20 19:00:21belopolskysetnosy: + belopolsky
messages: + msg110948
2010-07-20 18:04:34amaury.forgeotdarcsetmessages: + msg110939
2010-07-20 18:04:07amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg110937
2010-07-20 15:26:57eric.araujosetnosy: + eric.araujo
2010-07-20 11:28:40djccreate