classification
Title: PyModule_GetState doesn't work with LazyLoader
Type: Stage:
Components: C API Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, corona10, petr.viktorin, shihai1991
Priority: normal Keywords:

Created on 2020-09-16 13:23 by petr.viktorin, last changed 2020-09-25 13:08 by corona10.

Messages (2)
msg376991 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2020-09-16 13:23
One underlying issue from bpo-41631 is that importlib.util.LazyLoader produces modules that are only fully loaded after an attribute is accessed (through __getattribute__).

C-API functions like PyModule_GetState, PyModule_GetDict and such do not get attributes, and when called on a _LazyModule, they might try to access uninitialized C-level state.

I see two possible ways to fix this:
- Make PyModule_GetState, etc. trigger a full load. Since this is C-API, it would need a fast (no Python code) way either to do the full load or to detect _LazyModule.
- Make LazyLoader eager for either all extension & builtin modules, or for modules that have C-level storage. The latter would need a way to pass info from PyModuleDef to the loader.


Brett, does that analysis make sense from the importlib POV?
msg377009 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-09-16 18:33
Probably making LazyLoader skip being lazy for non-source modules probably makes the most sense and would be easiest to implement since it's an explicit opt-out.
History
Date User Action Args
2020-09-25 13:08:51corona10setnosy: + corona10
2020-09-17 16:46:44shihai1991setnosy: + shihai1991
2020-09-16 18:33:29brett.cannonsetmessages: + msg377009
2020-09-16 13:23:38petr.viktorincreate