classification
Title: yield in list comprehensions possibly broken in 3.0
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.0
process
Status: closed Resolution: wont fix
Dependencies: Superseder: yield expression inside generator expression does nothing
View: 10544
Assigned To: Nosy List: benjamin.peterson, brett.cannon, erickt
Priority: normal Keywords:

Created on 2008-07-03 06:28 by erickt, last changed 2010-12-08 20:12 by terry.reedy. This issue is now closed.

Messages (3)
msg69168 - (view) Author: Erick Tryzelaar (erickt) Date: 2008-07-03 06:28
This may be a known consequence of python 3.0, but I couldn't find any 
reference to it, nor a test case that covers it. Here's a valid use of yield 
in 2.5.1:

>>> def foo():
...   x=[(yield x) for x in 1,2,3]
...   yield 5
...   yield x
>>> x=foo()
>>> x.next()
1
>>> x.send(6)
2
>>> x.send(7)
3
>>> x.send(8)
5
>>> x.send(9)
[6, 7, 8]
>>> x.send(10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration


But in python 3.0, this code results in:

>>> def foo():
...   x=[(yield x) for x in (1,2,3)]
...   yield 5
...   yield x
>>> x=foo()
>>> next(x)
5
>>> x.send(6)
<generator object <listcomp> at 0x3678f0>
>>> x.send(7)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration


Looking further, it seems that this is a comprehension:

>>> def foo(): [(yield 5)]
>>> type(foo())
<class 'generator'>

Whereas this is not:

>>> def foo(): [(yield 5) for x in range(3)]
>>> type(foo())
<class 'NoneType'>


Is this expected behavior?
msg69195 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-07-03 12:54
This is because list comprehensions are implemented as functions in 3.0
so their scope is not leaked out to the rest of the function.

Here is the bytecode for 2.6:
>>> dis.dis(f)
  2           0 BUILD_LIST               0
              3 DUP_TOP             
              4 STORE_FAST               0 (_[1])
              7 LOAD_CONST               5 ((1, 2, 3))
             10 GET_ITER            
        >>   11 FOR_ITER                14 (to 28)
             14 STORE_FAST               1 (x)
             17 LOAD_FAST                0 (_[1])
             20 LOAD_FAST                1 (x)
             23 YIELD_VALUE         
             24 LIST_APPEND         
             25 JUMP_ABSOLUTE           11
        >>   28 DELETE_FAST              0 (_[1])
             31 STORE_FAST               1 (x)

  3          34 LOAD_CONST               4 (5)
             37 YIELD_VALUE         
             38 POP_TOP             

  4          39 LOAD_FAST                1 (x)
             42 YIELD_VALUE         
             43 POP_TOP             
             44 LOAD_CONST               0 (None)
             47 RETURN_VALUE        

and here it is for 3.0:
>>> dis.dis(f)
  2           0 LOAD_CONST               1 (<code object <listcomp> at
0x740770, file "<stdin>", line 2>) 
              3 MAKE_FUNCTION            0 
              6 LOAD_CONST               6 ((1, 2, 3)) 
              9 GET_ITER             
             10 CALL_FUNCTION            1 
             13 STORE_FAST               0 (x) 

  3          16 LOAD_CONST               5 (5) 
             19 YIELD_VALUE          
             20 POP_TOP              

  4          21 LOAD_FAST                0 (x) 
             24 YIELD_VALUE          
             25 POP_TOP              
             26 LOAD_CONST               0 (None) 
             29 RETURN_VALUE
msg69210 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-07-03 17:30
Yes, this change in semantics is expected. Closing as "won't fix".
History
Date User Action Args
2010-12-08 20:12:02terry.reedysetsuperseder: yield expression inside generator expression does nothing
2008-07-03 17:30:59brett.cannonsetstatus: open -> closed
resolution: wont fix
messages: + msg69210
nosy: + brett.cannon
2008-07-03 12:54:10benjamin.petersonsettype: behavior
messages: + msg69195
nosy: + benjamin.peterson
2008-07-03 06:28:18ericktcreate