Since _imp.init_builtin and _imp.load_dynamic don't take in a module to load, need to create new functions which do. Afterwards can deprecate init_builtin, load_dynamic, and init_frozen (the latter having been worked around thanks to get_frozen_object).

This is going to have to wait until Python 3.5, so I'm going to back out the exec_module() aspects of BuiltinImporter and ExtensionFileLoader. As Nick has pointed out previously, we are going to need to change the init function signature of extension modules to accept the module to initialize (or something). This also extends to built-in modules as on Windows all extension modules are compiled in, so there is no real distinction between the two scenarios, so this can't be only partially solved for built-ins but not extension modules.

This is another reason why we will need to go with a soft deprecation of load_module() in Python 3.4 and hopefully get all of this straightened out in Python 3.5 so we can introduce a hard deprecation.

And just in case we end up having to hack our way around the call signature, PyModule_Create() could be tweaked to pull modules from a cache that can be seeded with the module that is to be used for the initialization. That way call signatures don't have to change but what module gets used can be controlled.

New changeset b5bbd47a9bc4 by Brett Cannon in branch 'default':
Issue #19698: Remove exec_module() from the built-in and extension
http://hg.python.org/cpython/rev/b5bbd47a9bc4

Is the problem here that builtins (and extension modules) don't necessarily obey the rules to the letter?  load_module() is supposed to use whatever is in sys.modules. [1]  (BuiltinImporter.load_module() is essentially a synonym for _imp.init_builtin(), right?)

So in each PyInit_*() the code should be using the module out of sys.modules...  That sounds like a bug and I expect that the builtin modules and most extension modules don't comply.

That said, I realize the spirit of the load_module() rule is to support reloading, which is mostly not applicable to builtins and extension modules.  The code you reverted would rely on enforcing the rule outside the implicit applicability.

I'll be glad when we've resolved the extension module API changes that support exec_module()!

[1] http://docs.python.org/dev/library/importlib.html#importlib.abc.Loader.load_module

There are assorted shenanigans in the dynamic module loading code that make
me think we should leave the associated loaders alone for now. Running
PyInit_* more than once isn't permitted by default, so reloading is a
no-op, and "loading again" *copies* the existing module. But modules can
opt in to allowing reinitialization by declaring a module state size of
zero :)

It's a solvable problem, but not an urgent one, and hence better addressed
when we're redesigning the associated C APIs for a PEP 451 based import
system.

I would still like to get this solved for Python 3.5. Should we hash out a solution at PyCon?

Yes, if we don't get to it beforehand. I'd still like to take the draft Create/Exec C level hook design I came up with and turn it into a PEP, but I don't know when I'll get time.

Maybe I should just put that together as a (very) rough draft and lob it at import-sig? Then we'll have a concrete base for discussion at PyCon, and someone may be able to put together a draft implementation while you, me & Eric are all in the same place at the same time.

sgtm

Same here.

Turns out I had written up a recap of the PEP 451 C extension support status back in July, so I just resent that to import-sig.

At this point both FrozenImporter and BuiltinImporter support exec_module(), so the only thing left to do is to document when FrozenImporter and WindowsRegistryFinder gained their exec_module() implementations (BuiltinImporter already has a note about it).