Currently, the runpy module imports many modules. runpy is used by "python3 -m module". I propose to attempt to reduce the number of imports to reduce Python startup time.


With my local changes, I reduce Python startup time from 24 ms to 18 ms:

Mean +- std dev: [ref] 24.3 ms +- 0.2 ms -> [patch] 18.0 ms +- 0.3 ms: 1.35x faster (-26%)

Timing measured by:

  ./python -m venv env
  python -m pyperf command -v -o patch.json -- env/bin/python -m empty


Currently, runpy imports +21 modules:

* ./python mod.py: Total 33
* ./python -m mod: Total 54 (+21)


Example with attached mod.py:

$ ./python -m mod
['__main__',
 '_abc',
 '_codecs',
 '_collections',
 '_collections_abc',
 '_frozen_importlib',
 '_frozen_importlib_external',
 '_functools',
 '_heapq',
 '_imp',
 '_io',
 '_locale',
 '_operator',
 '_signal',
 '_sitebuiltins',
 '_sre',
 '_stat',
 '_thread',
 '_warnings',
 '_weakref',
 '_weakrefset',
 'abc',
 'builtins',
 'codecs',
 'collections',
 'collections.abc',
 'contextlib',
 'copyreg',
 'encodings',
 'encodings.aliases',
 'encodings.ascii',
 'encodings.latin_1',
 'encodings.utf_8',
 'enum',
 'functools',
 'genericpath',
 'heapq',
 'importlib',
 'importlib._bootstrap',
 'importlib._bootstrap_external',
 'importlib.abc',
 'importlib.machinery',
 'importlib.util',
 'io',
 'itertools',
 'keyword',
 'marshal',
 'operator',
 'os',
 'os.path',
 'pkgutil',
 'posix',
 'posixpath',
 're',
 'reprlib',
 'runpy',
 'site',
 'sre_compile',
 'sre_constants',
 'sre_parse',
 'stat',
 'sys',
 'time',
 'types',
 'typing',
 'typing.io',
 'typing.re',
 'warnings',
 'weakref',
 'zipimport']
Total 70

My local changes removed the following imports from runpy:

- '_heapq',
- '_locale',
- '_sre',
- 'collections.abc',
- 'copyreg',
- 'enum',
- 'heapq',
- 'importlib.abc',
- 'itertools',
- 're',
- 'sre_compile',
- 'sre_constants',
- 'sre_parse',
- 'typing',
- 'typing.io',
- 'typing.re',

> Currently, runpy imports +21 modules:
> * ./python mod.py: Total 33
> * ./python -m mod: Total 54 (+21)

Oops sorry, that's with my local changes!

Currently, runpy imports not less than 37 modules:

* ./python mod.py: Total 33
* ./python -m mod: Total 70 (+37)

See also bpo-40275: "test.support has way too many imports".

I created 3 PRs. I have a few more local branches to avoid types and itertools imports. I may create PRs for these ones as well.

New changeset 7824cc05bfe7f8181b21848a52007ddaf5612b9b by Victor Stinner in branch 'master':
bpo-41006: collections imports lazily heap (GH-20940)
https://github.com/python/cpython/commit/7824cc05bfe7f8181b21848a52007ddaf5612b9b

New changeset 98ce7b107e6611d04dc35a4f5b02ea215ef122cf by Victor Stinner in branch 'master':
bpo-41006: pkgutil imports lazily re (GH-20939)
https://github.com/python/cpython/commit/98ce7b107e6611d04dc35a4f5b02ea215ef122cf

New changeset 9e09849d20987c131b28bcdd252e53440d4cd1b3 by Victor Stinner in branch 'master':
bpo-41006: importlib.util no longer imports typing (GH-20938)
https://github.com/python/cpython/commit/9e09849d20987c131b28bcdd252e53440d4cd1b3

I close the issue. Making more imports lazy doesn't bring much benefit.

--

With the 3 changes, runpy now imports 55 modules, instead of 70.

The startup time is 1.3x faster, 18 ms instead of 24 ms:

Mean +- std dev: [before] 23.7 ms +- 0.4 ms -> [after] 17.8 ms +- 0.6 ms: 1.33x faster (-25%)

--

Avoiding itertools and types doesn't bring much benefit:

Mean +- std dev: [after] 17.8 ms +- 0.6 ms -> [WIP] 17.2 ms +- 0.4 ms: 1.03x faster (-3%)

New changeset 4c18fc8f1def7030e5ec5d1ffb9355d7453dc408 by Victor Stinner in branch 'master':
bpo-41006: Document the runpy optimization (GH-20953)
https://github.com/python/cpython/commit/4c18fc8f1def7030e5ec5d1ffb9355d7453dc408

New changeset 1bf7959dce0597e312c6f35476a7cc957fd0323c by Victor Stinner in branch 'master':
bpo-41006: Remove init_sys_streams() hack (GH-20954)
https://github.com/python/cpython/commit/1bf7959dce0597e312c6f35476a7cc957fd0323c

New changeset 2c2a4f3d8545784c6e4ca8128bfc706916080712 by Victor Stinner in branch 'master':
bpo-41006: What's New: less => fewer modules (GH-20955)
https://github.com/python/cpython/commit/2c2a4f3d8545784c6e4ca8128bfc706916080712