classification
Title: Speedup sysconfig startup
Type: performance Stage:
Components: Versions: Python 3.3
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, eric.araujo, eric.smith, eric.snow, neologix, pitrou, tarek, vstinner
Priority: normal Keywords: patch

Created on 2012-02-19 23:47 by vstinner, last changed 2013-01-03 02:06 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
sysconfig_parser-2.patch vstinner, 2012-02-21 23:21 review
Messages (5)
msg153735 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2012-02-19 23:47
On my laptop, start Python compiled in debug mode takes 600 ms. Half of this time is spend in the site module. And most of this time is spend in load the sysconfig module, which parse sysconfig.cfg, just to get the user site packages directory.

Attached patch adds a copy of configparser.RawConfigParser, specialized to parse sysconfig.cfg. Using this patch, Python startup is 25% faster (I didn't check in release mode).
msg153737 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2012-02-19 23:48
To speed up python -s, the following patch avoids loading the sysconfig module:

diff --git a/Lib/site.py b/Lib/site.py
--- a/Lib/site.py
+++ b/Lib/site.py
@@ -518,7 +518,8 @@ def main():
     known_paths = removeduppaths()
     if ENABLE_USER_SITE is None:
         ENABLE_USER_SITE = check_enableusersite()
-    known_paths = addusersitepackages(known_paths)
+    if ENABLE_USER_SITE:
+        known_paths = addusersitepackages(known_paths)
     known_paths = addsitepackages(known_paths)
     if sys.platform == 'os2emx':
         setBEGINLIBPATH()

I don't know if this patch is correct.
msg153738 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-02-20 00:21
The site.getusersitepackages, site.addusersitepackages and co. functions all call one function which makes sure site.USER_SITE is set according to envvars and command-line options; under python -s, addusersitepackages will not add the user site dir to sys.path, but it will cause site.USER_SITE to be set (to False), so that’s why I can’t be sure that your patch does not change behavior.  test_site has good coverage for IIRC (just not sure if it covers starting python -s/-S and then importing site, and python -s/-S then import site then call site.main).

I regret that all these site functions are public, now we can’t simplify them to have the setting of site.USER_SITE in only one place.
msg153914 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2012-02-21 23:21
Updated patch: continue to simplify the config parser. Using this patch, Python startup is ~20% faster on my computer.

Use http://bugs.python.org/file24447/bench_startup.py to measure startup time.
msg178898 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-01-03 02:06
I'm not really convinced by my patch. It looks like a quick hack. Nick's PEP 432 looks more promising (in speed), simple and safer. So I prefer to close this issue.
History
Date User Action Args
2013-01-03 02:06:47vstinnersetstatus: open -> closed
resolution: rejected
messages: + msg178898
2012-10-02 18:39:42pitrouunlinkissue16101 dependencies
2012-10-02 13:08:14brett.cannonlinkissue16101 dependencies
2012-02-21 23:21:12vstinnersetfiles: - sysconfig_parser.patch
2012-02-21 23:21:05vstinnersetfiles: + sysconfig_parser-2.patch

messages: + msg153914
2012-02-20 18:18:06eric.snowsetnosy: + eric.snow
2012-02-20 00:22:50Arfreversetnosy: + Arfrever
2012-02-20 00:21:51eric.araujosetmessages: + msg153738
2012-02-19 23:58:44eric.smithsetnosy: + eric.smith
2012-02-19 23:48:54vstinnersetnosy: + tarek, eric.araujo
messages: + msg153737
2012-02-19 23:47:14vstinnercreate