classification
Title: itertools.chain.from_iterable doesn't stop
Type: behavior Stage:
Components: Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: davidhalter, gwrtheyrn, mark.dickinson, pyos, r.david.murray, rhettinger
Priority: normal Keywords:

Created on 2012-12-27 12:30 by davidhalter, last changed 2012-12-28 17:04 by mark.dickinson. This issue is now closed.

Messages (8)
msg178296 - (view) Author: David Halter (davidhalter) Date: 2012-12-27 12:30
The following lines run infinitely:

b = [1]
import itertools
b += itertools.chain.from_iterable([c] for c in b)

In my opinion this is a bug. Clearly you could say that this is an implementation detail. But afaik this is not documented and very misleading.

BTW: In PyPy b would be [1, 1].
msg178298 - (view) Author: (pyos) Date: 2012-12-27 13:34
I don't think this is a bug. `b += iter(b)` and `for c in b: b.append(c)` work the same way. Also, the tutorial makes it clear that you should duplicate the list if you modify it inside a loop; in this case, this can be done either by iterating over `b[:]` instead of just `b` or using a list comprehension instead of a generator.
msg178299 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-12-27 13:48
Yes, the behavior here is "undefined" at the language level.  And as pyos says, technically it is documented: if you mutate the object over which you are iterating inside the iteration context (a for loop is just one example of same), the behavior is explicitly undefined.  I agree that it is not *obvious* that that applies to chain.from_iterable in this context, but I'm not sure how we'd improve the docs to cover it.
msg178367 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2012-12-28 07:22
This isn't a bug.  It is an intended feature that chain.from_iterable evaluates lazily (and is documented as such).  The pure python equivalent in the docs behaves the same as the C version does.

Also, it is a long standing feature of lists that you can loop over them while mutating them (I've seen more than a few pieces of code that take advantage of that feature).
msg178390 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-12-28 16:43
If it is a feature, then is it documented in the language reference and is this actually a bug in PyPy (it sounds like it is)?
msg178393 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-12-28 16:52
It looks to me as though this has nothing to do with itertools  In CPython 2.7:


Python 2.7.3 |EPD 7.3-1 (32-bit)| (default, Apr 12 2012, 11:28:34) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "credits", "demo" or "enthought" for more information.
>>> b = [1]
>>> b += (x for x in b)   # runs until Ctrl-C
^CTraceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <genexpr>
KeyboardInterrupt


In PyPy:

Python 2.7.2 (341e1e3821fff77db3bb5cdb7a4851626298c44e, Jun 09 2012, 14:24:15)
[PyPy 1.9.0] on darwin
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``the world doesn't want us to
know''
>>>> b = [1]
>>>> b += (x for x in b)  # Returns immediately.
>>>> b
[1, 1]

So it seems that PyPy is building the RHS before appending it to b.  That looks like a bug in PyPy to me.
msg178394 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-12-28 16:54
And here's a non-infinite example where CPython and PyPy differ.

Python 2.7.3 |EPD 7.3-1 (32-bit)| (default, Apr 12 2012, 11:28:34) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "credits", "demo" or "enthought" for more information.
>>> b = [1]
>>> b += (x+1 for x in b if x < 5)
>>> b
[1, 2, 3, 4, 5]
>>> 
iwasawa:cpython mdickinson$ pypy-c
-bash: pypy-c: command not found
iwasawa:cpython mdickinson$ /opt/local/bin/pypy-c
Python 2.7.2 (341e1e3821fff77db3bb5cdb7a4851626298c44e, Jun 09 2012, 14:24:15)
[PyPy 1.9.0] on darwin
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``'that's definitely a case of
"uh????"'''
>>>> b = [1]
>>>> b += (x+1 for x in b if x < 5)
>>>> b
[1, 2]
msg178395 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-12-28 17:04
Opened https://bugs.pypy.org/issue1355 for this.
History
Date User Action Args
2012-12-28 17:04:56mark.dickinsonsetmessages: + msg178395
2012-12-28 16:54:38mark.dickinsonsetmessages: + msg178394
2012-12-28 16:52:10mark.dickinsonsetnosy: + mark.dickinson
messages: + msg178393
2012-12-28 16:43:22r.david.murraysetmessages: + msg178390
2012-12-28 07:22:40rhettingersetstatus: open -> closed
resolution: not a bug
messages: + msg178367
2012-12-28 07:15:45rhettingersetassignee: rhettinger

nosy: + rhettinger
2012-12-27 13:48:46r.david.murraysetnosy: + r.david.murray
messages: + msg178299
2012-12-27 13:34:10pyossetnosy: + pyos
messages: + msg178298
2012-12-27 12:32:44gwrtheyrnsetnosy: + gwrtheyrn
2012-12-27 12:30:02davidhaltercreate