New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
operator.py: move the Python implementation in the else block of try/except ImportError #63428
Comments
To speedup Python startup, it may be interesting to not create useless many functions and classes in operator.py: "from _operator import *" will remove them a few line later. What do you think of moving the Python implementation of the operator module inside in the else block of "try/except ImportError" section? See attached operator.py for an example. It adds an ugly level of indentation, but it's for performances! Another option is to add a _pyoperator module. |
Using the microbenchmark I get (standard version): ./python -m timeit "import sys; modname='operator'" "__import__(modname); del sys.modules[modname]" Victor's version: ./python -m timeit "import sys; modname='operator'" "__import__(modname); del sys.modules[modname]" Importing _operator directly: ./python -m timeit "import sys; modname='_operator'" "__import__(modname); del sys.modules[modname]" Extrapolating from what I did with decimal, I guess a _pyoperator |
To be fair, for the startup time I can't really detect any difference between importing _operator directly and the current setup. |
Attached builtin_operator.patch patch implements this option: operator.c becomes the main operator module, _pyoperator is the pure Python implementation (don't use "from _operator import *" anymore). With the patch: $ ./python -m timeit "import sys; modname='_pyoperator'" "__import__(modname); del sys.modules[modname]"
1000 loops, best of 3: 276 usec per loop
$ ./python -m timeit "import sys; modname='operator'" "__import__(modname); del sys.modules[modname]"
10000 loops, best of 3: 22.7 usec per loop The patch is huge because files are renamed: see builtin_operator_diff.patch for the diff. |
Without the patch: $ ./python -m timeit "import sys; modname='operator'" "__import__(modname); del sys.modules[modname]"
1000 loops, best of 3: 289 usec per loop
$ ./python -m timeit "import sys; modname='_operator'" "__import__(modname); del sys.modules[modname]"
10000 loops, best of 3: 21.4 usec per loop "import operator" is 12.7x faster (289 usec => 22.7 usec) with builtin_operator.patch. |
Why not: try: |
Let's try (I replaced operator.py with these 4 lines). $ ./python -m timeit "import sys; modname='operator'" "__import__(modname); del sys.modules[modname]; del sys.modules['_operator']"
10000 loops, best of 3: 165 usec per loop
$ ./python -m timeit "import sys; modname='operator'" "__import__(modname); del sys.modules[modname]"
10000 loops, best of 3: 136 usec per loop "import operator" is only 2x faster (289 usec => 136 usec). It's less interesting. And what would be the purpose of a file of 4 line which containing "import *"? Do you think that PyPy, IronPython and Jython will reuse such trampoline/wrapper? |
The real question for me is: why are you interested in speeding up the
To make it obvious that there are two implementations, one of which is |
I understand the desire to speed things up, but either the four-line facade |
"The real question for me is: why are you interested in speeding up the Alone, the gain is useless, but it's like the work done in Python 3.4 to avoid loading some modules at startup. The overall idea is to have a fast startup time. I heard that Python 3 startup time is a major blocker point for Mercurial for example. But maybe this specific issue is not worth the trouble. (Other parts should be optimized.) |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: