This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Faster list comprehensions
Type: Stage:
Components: Interpreter Core Versions: Python 2.5
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: nnorwitz Nosy List: collinwinter, nnorwitz, rhettinger
Priority: normal Keywords: patch

Created on 2006-03-03 12:41 by collinwinter, last changed 2022-04-11 14:56 by admin. This issue is now closed.

File name Uploaded Description Edit
listcomp.patch collinwinter, 2006-03-03 12:41 Patch against r42780
Messages (5)
msg49629 - (view) Author: Collin Winter (collinwinter) * (Python committer) Date: 2006-03-03 12:41
The attached patch results in a 16% speed increase for
list comprehensions. By changing how the LIST_APPEND
opcode works (which was previously totally unused),
it's possible to dramatically simplify the emitted
bytecode for listcomps.

More details are available on my blog:
msg49630 - (view) Author: Collin Winter (collinwinter) * (Python committer) Date: 2006-03-03 12:49
Logged In: YES 

I forgot to mention: the patch also includes updates to
things like Lib/test/ and Lib/ to make
them work with the updated LIST_APPEND opcode and code
msg49631 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2006-03-03 16:21
Logged In: YES 

Neal, did the LIST_APPEND generation get left out of the 
AST.  In Py2.4, is was working fine:

Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 
bit (Intel)] on win32
>>> from dis import dis
>>> dis(compile('[x for x in range(10) if x % 
2]', '', 'single'))
  1           0 BUILD_LIST               0
              3 DUP_TOP             
              4 STORE_NAME               0 (_[1])
              7 LOAD_NAME                1 (range)
             10 LOAD_CONST               0 (10)
             13 CALL_FUNCTION            1
             16 GET_ITER            
        >>   17 FOR_ITER                28 (to 48)
             20 STORE_NAME               2 (x)
             23 LOAD_NAME                2 (x)
             26 LOAD_CONST               1 (2)
             29 BINARY_MODULO       
             30 JUMP_IF_FALSE           11 (to 44)
             33 POP_TOP             
             34 LOAD_NAME                0 (_[1])
             37 LOAD_NAME                2 (x)
             40 LIST_APPEND         
             41 JUMP_ABSOLUTE           17
        >>   44 POP_TOP             
             45 JUMP_ABSOLUTE           17
        >>   48 DELETE_NAME              0 (_[1])
             51 PRINT_EXPR          
             52 LOAD_CONST               2 (None)
             55 RETURN_VALUE 
msg49632 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2006-03-03 17:23
Logged In: YES 

I believe the patch is not correct for comprehensions with 
multiple for-loops.  It needs to leave ceval.c, opcode.h, 
and exactly as the were in Py2.4.  Also, the 
code for ceval.c should only access the stack through the 
stack access macros and avoid derefencing the stack 
pointer directly.

The patch should be resubmitted to only modify compile.c 
and to produce the same code as in Py2.4 (see example 
below and note the inclusion of the DUP_TOP after the 
msg49633 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2006-03-03 20:30
Logged In: YES 

Raymond, I added back the LIST_APPEND opcode.  Collin, if
you have a further enhancements, please open a new patch. 
Closing this one.
Date User Action Args
2022-04-11 14:56:15adminsetgithub: 42972
2006-03-03 12:41:03collinwintercreate