It was privately reported to me that importlib.util.LazyLoader rejects using importlib.machinery.SourceFileLoader (or at least _frozen_importlib.SourceFileLoader). At least a test should be added for LazyLoader to make sure it will happily accept importlib.machinery.SourceFileLoader, and if it rejects the one from _frozen_importlib, tweak the test to accept loaders from there as well.

One way to possibly improve this is to remove the create_module() check and replace it with a warning if create_module() doesn't return None. Another option is to use an assert statement. The final option is to just drop the check entirely, although that can make debugging difficult.

I think I'm liking the following approach:

  if __debug__:
    hopefully_None = loader.create_module(spec)
    if hopefully_None is not None:
        warnings.warn("watch out!", ImportWarning)

Just stumbled on this very issue while trying to use LazyLoader.

My current plan is to simply remove the check in 3.5 -- the docs say it's ignored so I don't think it will hurt anything -- and add the warning I proposed in 3.6. What do you think, Antoine?

By the way does this mean the LazyLoader won't work with ExtensionFileLoader? That would reduce its usefulness quite a bit.

I don't know what the impact of the error is, but it seems like at least the default loader classes should be able to work with LazyLoader...

You're right, it won't work with extension modules based on how ExtensionFileLoader is structured.

New changeset 9f1e680896ef by Brett Cannon in branch '3.5':
Issue bpo-26186: Remove an invalid type check in
https://hg.python.org/cpython/rev/9f1e680896ef

New changeset 86fc6cdd65de by Brett Cannon in branch 'default':
Merge for issue bpo-26186
https://hg.python.org/cpython/rev/86fc6cdd65de

This has been resolved by removing the check (the docs have always said the method was ignored, so that will just continue). I also did separate commit to list that BuiltinImporter and ExtensionFileLoader won't work (they would need to be updated to return a module subclass to work).

I'm leaving this issue open to decide if I want to add an explicit check in importlib.util.LazyLoader.create_module() to verify that None is returned in Python 3.5 (I probably will, but I want to think about on it for a little bit).

I actually think this can be generalized thanks to some tweaks made in Python 3.6 which will lead to it working with extension modules.

I need to double-check, but I think I can:

• Eliminate the _Module class (and thus have _LazyModule inherit types.ModuleType directly)
• Call create_module() and see if it returns something
• Use spec.loader_state to store the original value of module.__class__ (probably be simply storing __class__ in the dict of values)
• Just blindly try to set module.__class__ to _LazyModule and let the exception propagate if it fails
• In _LazyModule.__getattribute__, temporarily assign the type to types.ModuleType to shut off __getattribute__() before assigning the proper value

New changeset 7af4b3ad75b7 by Brett Cannon in branch 'default':
Issue bpo-26186: Remove the restriction that built-in and extension
https://hg.python.org/cpython/rev/7af4b3ad75b7

The restriction of lazily loading built-ins and extensions is now lifted!