classification
Title: operator.concat/iconcat could only work if left operand is a sequence
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: rhettinger, serhiy.storchaka, vstinner, xiang.zhang, zach.ware
Priority: normal Keywords:

Created on 2017-01-03 09:58 by xiang.zhang, last changed 2017-03-18 06:26 by serhiy.storchaka.

Messages (6)
msg284546 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-01-03 09:58
As the title, but I think it should also continue if only the right operand is a sequence since the right operand could get a __radd__ to do the concatenation:

>>> class S:
...     def __init__(self):
...             self.v = list(range(10))
...     def __radd__(self, other):
...             print('in __radd__')
...             self.v.extend(other)
...             return self.v
...     def __getitem__(self, idx):
...             return self.v[idx]
>>> def i():
...     yield from range(10, 20)
>>> i() + S()
in __radd__
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
>>> operator.concat(i(), S())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'generator' object can't be concatenated
>>> a = i()
>>> a += S()
in __radd__
>>> a
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
>>> a = i()
>>> operator.iconcat(a, S())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'generator' object can't be concatenated
msg284547 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-01-03 10:17
I don't think it should. You can use operator.add() if you need to fall back to a __radd__ of the right operand.

operator.concat() is Python API to the sq_concat slot. It falls back to __add__() just because instances of user classes defining an __add__() method only have an nb_add slot, not an sq_concat slot.

Third-party classes (maybe NumPy arrays, I don't know) can have different implementations of sq_concat and nb_add.
msg284549 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-01-03 11:07
> You can use operator.add() if you need to fall back to a __radd__ of the right operand.

IMHO for sequences when defining __add__, __radd__, there is no difference between addition and concatenation, just as https://docs.python.org/3/reference/datamodel.html#emulating-container-types states. So it's confusing for me why must I use operator.add not operator.concate?

> operator.concat() is Python API to the sq_concat slot. It falls back to __add__() just because instances of user classes defining an __add__() method only have an nb_add slot, not an sq_concat slot.

More like a implementation detail for me. Users writing only Python won't realize such things.

> Third-party classes (maybe NumPy arrays, I don't know) can have different implementations of sq_concat and nb_add.

Does this matter? Anyway operator.concat will fall to '+'. My intent is proposing something like:

    if not hasattr(a, '__getitem__') and not hasattr(b, '__getitem__')
msg284577 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-01-03 15:20
> More like a implementation detail for me.

IMHO it's a deliberate choice that the operator module behaves
differently for concat() and add().
msg284581 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-01-03 16:36
Actually this issue was discussed when import the pure Python implementation in #16694. There seems once an effort checking also the right operand but was later removed, but I couldn't understand why (though there is an explanation, but it's more about why removing __len__ checking). :-(
msg289792 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-03-18 03:52
I think it is too late in the game to change the semantics of this function.  Changing it now will only break code.  The time for API discussion is before a release, not after.  Also, we have no indication from actual users that there is an actual problem here.
History
Date User Action Args
2017-03-18 06:26:19serhiy.storchakasetpull_requests: - pull_request590
2017-03-18 03:52:32rhettingersetmessages: + msg289792
versions: - Python 3.5, Python 3.6
2017-03-17 21:00:33larrysetpull_requests: + pull_request590
2017-01-03 16:36:19xiang.zhangsetnosy: + rhettinger
messages: + msg284581
2017-01-03 15:20:56vstinnersetmessages: + msg284577
2017-01-03 11:07:48xiang.zhangsetmessages: + msg284549
2017-01-03 10:17:17serhiy.storchakasetmessages: + msg284547
2017-01-03 10:16:10xiang.zhangsetnosy: + zach.ware, serhiy.storchaka
2017-01-03 10:11:29vstinnersetnosy: + vstinner
2017-01-03 09:58:30xiang.zhangcreate