This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Implement _imp.exec_builtin and exec_dynamic
Type: Stage: needs patch
Components: Interpreter Core Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brett.cannon Nosy List: Arfrever, brett.cannon, docs@python, eric.snow, miss-islington, ncoghlan, python-dev
Priority: low Keywords: easy

Created on 2013-11-22 16:27 by brett.cannon, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19158 merged brett.cannon, 2020-03-25 18:16
PR 19164 merged miss-islington, 2020-03-25 18:58
PR 19165 merged miss-islington, 2020-03-25 18:58
Messages (14)
msg203798 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-22 16:27
Since _imp.init_builtin and _imp.load_dynamic don't take in a module to load, need to create new functions which do. Afterwards can deprecate init_builtin, load_dynamic, and init_frozen (the latter having been worked around thanks to get_frozen_object).
msg204726 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-29 15:18
This is going to have to wait until Python 3.5, so I'm going to back out the exec_module() aspects of BuiltinImporter and ExtensionFileLoader. As Nick has pointed out previously, we are going to need to change the init function signature of extension modules to accept the module to initialize (or something). This also extends to built-in modules as on Windows all extension modules are compiled in, so there is no real distinction between the two scenarios, so this can't be only partially solved for built-ins but not extension modules.

This is another reason why we will need to go with a soft deprecation of load_module() in Python 3.4 and hopefully get all of this straightened out in Python 3.5 so we can introduce a hard deprecation.

And just in case we end up having to hack our way around the call signature, PyModule_Create() could be tweaked to pull modules from a cache that can be seeded with the module that is to be used for the initialization. That way call signatures don't have to change but what module gets used can be controlled.
msg204735 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-11-29 16:00
New changeset b5bbd47a9bc4 by Brett Cannon in branch 'default':
Issue #19698: Remove exec_module() from the built-in and extension
http://hg.python.org/cpython/rev/b5bbd47a9bc4
msg205355 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-12-06 06:35
Is the problem here that builtins (and extension modules) don't necessarily obey the rules to the letter?  load_module() is supposed to use whatever is in sys.modules. [1]  (BuiltinImporter.load_module() is essentially a synonym for _imp.init_builtin(), right?)

So in each PyInit_*() the code should be using the module out of sys.modules...  That sounds like a bug and I expect that the builtin modules and most extension modules don't comply.

That said, I realize the spirit of the load_module() rule is to support reloading, which is mostly not applicable to builtins and extension modules.  The code you reverted would rely on enforcing the rule outside the implicit applicability.

I'll be glad when we've resolved the extension module API changes that support exec_module()!

[1] http://docs.python.org/dev/library/importlib.html#importlib.abc.Loader.load_module
msg205358 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-12-06 08:33
There are assorted shenanigans in the dynamic module loading code that make
me think we should leave the associated loaders alone for now. Running
PyInit_* more than once isn't permitted by default, so reloading is a
no-op, and "loading again" *copies* the existing module. But modules can
opt in to allowing reinitialization by declaring a module state size of
zero :)

It's a solvable problem, but not an urgent one, and hence better addressed
when we're redesigning the associated C APIs for a PEP 451 based import
system.
msg232569 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2014-12-12 17:40
I would still like to get this solved for Python 3.5. Should we hash out a solution at PyCon?
msg232612 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-12-13 14:53
Yes, if we don't get to it beforehand. I'd still like to take the draft Create/Exec C level hook design I came up with and turn it into a PEP, but I don't know when I'll get time.

Maybe I should just put that together as a (very) rough draft and lob it at import-sig? Then we'll have a concrete base for discussion at PyCon, and someone may be able to put together a draft implementation while you, me & Eric are all in the same place at the same time.
msg232623 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2014-12-13 23:24
sgtm
msg232701 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-12-16 02:49
Same here.
msg232743 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-12-16 13:54
Turns out I had written up a recap of the PEP 451 C extension support status back in July, so I just resent that to import-sig.
msg258962 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-01-26 17:14
At this point both FrozenImporter and BuiltinImporter support exec_module(), so the only thing left to do is to document when FrozenImporter and WindowsRegistryFinder gained their exec_module() implementations (BuiltinImporter already has a note about it).
msg365013 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-03-25 18:57
New changeset 302e5a8f79514fd84bafbc44b7c97ec636302322 by Brett Cannon in branch 'master':
bpo-19698: Document when importlib.machinery.FrozenImporter gained spec-related methods (GH-19158)
https://github.com/python/cpython/commit/302e5a8f79514fd84bafbc44b7c97ec636302322
msg365097 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-03-26 17:15
New changeset 8dad09a2242194a4101374b91ce87bb47a487cf7 by Miss Islington (bot) in branch '3.8':
bpo-19698: Document when importlib.machinery.FrozenImporter gained spec-related methods (GH-19158) (#19164)
https://github.com/python/cpython/commit/8dad09a2242194a4101374b91ce87bb47a487cf7
msg365098 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-03-26 17:15
New changeset 2929bfb973da249ed4f056c88a69bcc48e5a4843 by Miss Islington (bot) in branch '3.7':
bpo-19698: Document when importlib.machinery.FrozenImporter gained spec-related methods (GH-19158) (GH-19165)
https://github.com/python/cpython/commit/2929bfb973da249ed4f056c88a69bcc48e5a4843
History
Date User Action Args
2022-04-11 14:57:54adminsetgithub: 63897
2020-03-26 17:15:25brett.cannonsetmessages: + msg365098
2020-03-26 17:15:10brett.cannonsetmessages: + msg365097
2020-03-25 18:59:21brett.cannonsetkeywords: - patch
status: open -> closed
resolution: fixed
stage: patch review -> needs patch
2020-03-25 18:58:07miss-islingtonsetpull_requests: + pull_request18525
2020-03-25 18:58:00miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request18524
2020-03-25 18:57:54brett.cannonsetmessages: + msg365013
2020-03-25 18:16:51brett.cannonsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request18518
2020-03-06 20:44:51brett.cannonsetassignee: docs@python -> brett.cannon
2016-01-26 17:14:46brett.cannonsetpriority: critical -> low

assignee: docs@python
versions: + Python 3.6
keywords: + easy
nosy: + docs@python

messages: + msg258962
stage: test needed -> needs patch
2014-12-16 13:54:32ncoghlansetmessages: + msg232743
2014-12-16 02:49:31eric.snowsetmessages: + msg232701
2014-12-13 23:24:13brett.cannonsetmessages: + msg232623
2014-12-13 14:53:56ncoghlansetmessages: + msg232612
2014-12-12 17:40:19brett.cannonsetmessages: + msg232569
2013-12-11 17:16:18eric.snowunlinkissue18864 dependencies
2013-12-06 08:33:33ncoghlansetmessages: + msg205358
2013-12-06 06:35:08eric.snowsetmessages: + msg205355
2013-11-29 16:00:21python-devsetnosy: + python-dev
messages: + msg204735
2013-11-29 15:18:15brett.cannonsetpriority: normal -> critical

messages: + msg204726
versions: + Python 3.5, - Python 3.4
2013-11-23 19:50:26Arfreversetnosy: + Arfrever
2013-11-22 16:28:05brett.cannonlinkissue18864 dependencies
2013-11-22 16:27:55brett.cannoncreate