New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
_osx_support imports many modules #63524
Comments
On Mac OS X the site module imports sysconfig which imports _osx_support which in turn imports several more modules like the re module. More modules == slower startup. See bpo-19205 for background information on the issue. |
Actually Lib/_osx_support.py directly imports only five modules: os, re and sys at top level, and contextlib and tempfile in _read_output(). These last two aren't really needed at all, and there's no point even trying to avoid importing os and sys, so the only real problem is re which imports lots of other modules. It might be possible to avoid using re (the regular expressions don't look very complicated) but I noticed that some functions are only ever called from distutils, so maybe they should be moved from _osx_support to distutils._osx_support before doing anything else. I'm willing to do that if it's considered a good idea. |
For distutils function you can move the import of re inside the function body. It's not elegant but it does the trick w/o much slowdown or inconveniences. |
Yes, there are various tricks to be done. I have another solution in mind that should eliminate the use re altogether for most cases. (BTW, _osx_support was implemented to centralize functions in anticipation of the introduction of distutils2/packaging in Python 3.3, along with distutils.) |
The use of context lib in _read_output should not be necessary anymore as file objects and NamedTemporaryFile objects already are context managers with the right semantics. I'm not sure how to avoid the import of tempfile other than adding a minimal implementation of tempfile.mkstemp to _osx_support, in particular because the fallback code in _osx_support is insecure: It uses a named temporary file in /tmp with builtin.open and because the name of the file is predictable there is a small risk of overwriting arbitrary files when an attacker has access to /tmp. I guess the fallback is there to use during bootstrap, it should really be avoided afterwards. BTW. A small unscientific test on my laptop didn't see any differences between the regular _osx_support and a version where "import re" was moved inside the functions that use that module. But: that's on a fast laptop with SSD for storage, there could easily be a difference on systems with slower storage. Ned: do you remember what your idea was w.r.t. avoid the use of re? I guess its easy enough to replace the current re-using code by code that only uses str methods, but your phrasing seems to indicate another plan. |
FYI, bpo-29585 removes _osx_support dependency from site.py |
Done: Lib/site.py doesn't import _osx_support anymore. So I close this issue. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: