New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Speed up list comprehensions by preallocating the list where possible #58334
Comments
This adds a new opcode which for certain list comprehensions (ones with no if statements and only a single comprehension), preallocates the list to the appropriate size. Patch is against 2.7, because it was a bit easier. On: def f():
for i in range(10000):
[j for j in range(10000)]
f() Here's the speedup: alex@alex-gaynor-laptop:/tmp$ # Fresh 2.7 branch real 0m6.418s real 0m5.670s real 0m5.688s real 0m5.688s real 0m5.690s real 0m6.085s real 0m5.728s real 0m5.783s real 0m4.730s real 0m4.691s |
I think this proposal should be rejected for three reasons.
As I remember, Guido has rejected the idea of iterators having length information because in general it is not possible.
|
This seems like a reasonable optimization to me. |
You should try to port the patch to 3.3 and do some benchmark there. |
iter(range(10000)) should also see a speedup because range's iter supports __length_hint__ |
Could you run the benchamrks at http://hg.python.org/benchmarks/ Adding a new bytecode because it speeds up one 4 line program does not seem such a good idea. |
I believe this was implemented in bpo-33234 |
I don't think so. The bytecode in Python 3.9 still uses "BUILD_LIST 0": Python 3.9.0a4+ (heads/daemon_thread_runtime2-dirty:48652767d5, Mar 7 2020, 00:56:07)
>>> def f():
... for i in range(10000):
... [j for j in range(10000)]
...
>>> import dis; dis.dis(f)
(...) Disassembly of <code object <listcomp> at 0x7ffab2c9fd40, file "<stdin>", line 3>: |
Aah, thanks for the catcher Victor. Missed that this was about list /comprehensions/, not the list constructor. |
Isn't this the same as https://bugs.python.org/issue36551 ? |
Yes. |
Duplicated by https://bugs.python.org/issue36551 . |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: