classification
Title: list() != []
Type: behavior Stage: needs patch
Components: Documentation Versions: Python 3.3, Python 3.2
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: Peter.Norvig, cvrebert, docs@python, ezio.melotti, r.david.murray, rhettinger, terry.reedy
Priority: low Keywords:

Created on 2012-05-17 22:47 by Peter.Norvig, last changed 2012-05-20 07:14 by rhettinger.

Messages (5)
msg161023 - (view) Author: Peter Norvig (Peter.Norvig) Date: 2012-05-17 22:47
PEP 289 says "the semantic definition of a list comprehension in Python 3.0 will be equivalent to list(<generator expression>).  Here is a counterexample where they differ (tested in 3.2):

def five(x):
    "Generator yields the object x five times."
    for _ in range(5):
        yield x


# If we ask five() for 10 objects in a list comprehension,
# we get an error:

>>> F = five('x')
>>> [next(F) for _ in range(10)]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

# But if we ask five() for 10 objects in a list(generator expr),
# we get five objects, no  error:

>>> F = five('x')
>>> list(next(F) for _ in range(10))
['x', 'x', 'x', 'x', 'x']
msg161030 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-05-18 02:26
I think the behavior is correct.  next(x) is outside the for expression in the list comprehension, but 'list(x)' is an implicit 'for x in exp'.  So I believe it is the doc that needs amplification.  The PEP discussion is referring specifically to the leaking of the loop variable.
msg161033 - (view) Author: Peter Norvig (Peter.Norvig) Date: 2012-05-18 04:22
I agree with R. David Murray -- if "correct" means following the PEP 289 semantics, then

list(next(F) for _ in range(10))

should be the same as

def __gen(exp):
    for _ in exp:
        yield next(F)

list(__gen(iter(range(10))))

and indeed that is the case.  So the behvavior is correct and the documentation is both wrong, and rather informal/incomplete.
msg161084 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-05-18 20:50
It has been noted elsewhere (but I cannot find it) that 1) an uncaught StopIteration raised in an expression *within* a genexp is indistinguishable from the StopIteration raised *by* the genexp upon normal termination; and 2) this makes genexps different from otherwise equivalent comprehensions. The statement in PEP289 does not address this exceptional case.

Raymond, do you want to revise your PEP to mention this?

PEPs are, as their name says, proposals that express intention, not result, and especially not the result after continued patching. We do not usually usually go back and revise PEPs after they have been implemented. It would be an endless task. Hence, they are not current 'documentation' unless unless specifically referred from the docs. This one is not so referenced.

Perhaps PEP 0 should say that " Finished PEPs (done, implemented in code repository)" are historical documents and not current documentation.

I checked the entries for comprehensions and generator expressions and they do not claim equivalence, but do (briefly) describe the actual behavior. However, it is easy to miss that the entry in 5.2.8. Generator expressions implies the two facts above. Perhaps we should explicitly say something like:

If *expression* raises StopIteration, it will not be distinguished from the StopIteration raised upon normal termination of the generator, and it will make the generator expression act differently from the otherwise equivalent comprehension.
msg161187 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2012-05-20 07:14
I will be happy to clarify the PEP when I get a chance.
History
Date User Action Args
2012-05-20 07:14:03rhettingersetpriority: normal -> low
assignee: docs@python -> rhettinger
messages: + msg161187
2012-05-19 10:49:13cvrebertsetnosy: + cvrebert
2012-05-18 20:50:39terry.reedysetnosy: + terry.reedy, rhettinger, docs@python
messages: + msg161084

assignee: docs@python
components: + Documentation, - Interpreter Core
2012-05-18 04:22:26Peter.Norvigsetmessages: + msg161033
2012-05-18 02:26:49r.david.murraysetnosy: + r.david.murray
messages: + msg161030
2012-05-17 22:51:04ezio.melottisetnosy: + ezio.melotti
stage: needs patch

components: + Interpreter Core, - None
versions: + Python 3.3
2012-05-17 22:47:37Peter.Norvigcreate