This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Deep Lazy Imports - Interpreter-level deferred module loading
Type: enhancement Stage:
Components: Interpreter Core Versions: Python 3.11
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Kronuz, carljm, dino.viehland, itamaro, zsol
Priority: normal Keywords:

Created on 2022-03-08 19:32 by Kronuz, last changed 2022-04-11 14:59 by admin.

Messages (1)
msg414770 - (view) Author: Germán Méndez Bravo (Kronuz) * Date: 2022-03-08 19:32
As the size of a Python project increases, the number of modules and the complexity of its dependencies increases too, producing two problems in large codebases: increased risk of import cycles and slow start times due to the number of modules that tend to need getting loaded.

We propose implementing a robust and transparent lazy loader in CPython, based on an existing implementation we have in Cinder that proved to be extremely valuable to thousands of developers across Meta, and battle-tested in Instagram Server in production (

Our internal implementation is based on introducing deferred objects and converting top-level imported names to these deferred objects that are evaluated on first use. Enabling this across multiple types of workloads in Meta consistently shown improvements in startup times (up to 70%) and memory footprint (up to 40%), while virtually eliminating occurrences of import cycles.

Converting imported names to deferred objects is a semantic change, and the benefits are usually meaningful only at very large scale, so we propose not changing the default behavior, and making it possible for users to opt-in to this lazy loading mechanism via -x flags or environment variables.

This change would require related documentation changes that would cover the gotchas and edge cases, such as: some packages rely and import-time side-effects for correct operation; nested modules can’t be accessed unless imported explicitly; deferred loading may also defer exceptions from import time and first-access time which could be confusing.

The import-time side effects gotcha can be mitigated using eager-import lists and directives, though it could be desirable to use this as a forcing function to discourage package maintainers from relying on these side-effects.
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91119
2022-03-15 21:34:28zsolsetnosy: + zsol
2022-03-08 19:32:14Kronuzcreate