This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Using LazyLoader leads to AttributeError
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.10, Python 3.9, Python 3.8
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: KevKeating, epaine, eric.snow, ncoghlan
Priority: normal Keywords:

Created on 2020-11-05 22:14 by KevKeating, last changed 2022-04-11 14:59 by admin.

File name Uploaded Description Edit KevKeating, 2020-11-05 22:14 Files needed to reproduce issue
Messages (10)
msg380437 - (view) Author: Kevin Keating (KevKeating) Date: 2020-11-05 22:14
Steps to reproduce:

Create the following three files (or download the attached zip file, which contains these files):

    import foo
    from foo import a
    from foo import b



    import importlib.util
    import sys

    # implementation copied from
    def lazy_import(name):
        spec = importlib.util.find_spec(name)
        loader = importlib.util.LazyLoader(spec.loader)
        spec.loader = loader
        module = importlib.util.module_from_spec(spec)
        sys.modules[name] = module
        return module

    b = lazy_import("foo.b")


    def my_function():
        return "my_function"

and then run

Expected results

my_function should be printed to the terminal

Actual results

The following traceback is printed to the terminal

    Traceback (most recent call last):
      File "F:\Documents\lazy_import\", line 6, in <module>
    AttributeError: module 'foo' has no attribute 'b'

If you comment out "from foo import a" from, then the traceback doesn't occur and my_function gets printed.  Alternatively, if you move "from foo import a" after "from foo import b", then the traceback doesn't occur and my_function gets printed.  Adding "foo.b = b" before "print(foo.b.my_function())" will also fix the traceback.

A colleague of mine originally ran into this bug when writing unit tests for lazily imported code, since mock.patch("foo.b.my_function") triggers the same AttributeError.  I've reproduced this on Windows using both Python 3.8.3 and Python 3.9.0, and my colleague was using Python 3.8.3 on Mac.
msg380513 - (view) Author: E. Paine (epaine) * Date: 2020-11-07 17:37
Just checking: is this not because the lazy import should be in ``? (the code provided works fine with `a.b.my_function` on my system)
msg380617 - (view) Author: Kevin Keating (KevKeating) Date: 2020-11-09 21:31
An shouldn't be necessary.  If I comment out the 'b = lazy_import("foo.b")' line in (i.e. disable the lazy import), then the print statement works correctly as written without any other changes.

Also, I double checked with the colleague who originally ran into this issue, and it turns out he encountered the bug on Linux, not on Mac (still Python 3.8.3).
msg380618 - (view) Author: Kevin Keating (KevKeating) Date: 2020-11-09 21:53
My colleague just tested this on Mac and confirms that the bug also occurs there using Python 3.8.3.
msg380694 - (view) Author: E. Paine (epaine) * Date: 2020-11-10 19:02
In short, the module isn't being added to the package's namespace because we are directly modifying sys.modules (hence why the behaviour would be the same if we imported using `import foo.b` as `from foo import b`).

I personally prefer to use the metapath instead of modifying sys.modules but I agree that the given example should work when the lazy import is not in ``. The other solution is to modify the `LazyLoader` class to explicitly add the lazy module to the package's namespace (opinions?).
msg380718 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-11-10 21:57
The way import works,
msg380719 - (view) Author: Kevin Keating (KevKeating) Date: 2020-11-10 22:06
Brett, what do you mean by "the way import works"?  Is the difference between using LazyLoader and using a normal import intentional?
msg380720 - (view) Author: Kevin Keating (KevKeating) Date: 2020-11-10 22:09
One possible solution here would be to update the documentation at to either note the limitation or to modify the lazy_import function so that it adds the module to the package's namespace.  That's basically the workaround that we've been using.
msg380844 - (view) Author: E. Paine (epaine) * Date: 2020-11-12 19:23
Sorry Brett to readd you to the nosy for this, but we only got half a sentence in msg380718 (which is surely not what you intended?).

While I agree with you that this is not a bug, I do feel at least a note in the docs would be helpful to explain the implications of *adding* to sys.modules (the existing docs only mention replacing the dictionary or removing items from it).

Again, sorry to readd you to the nosy but Kevin's msg380719 was specifically for you.
msg380848 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-11-12 20:41
You can ignore the half sentence. I was contemplating closing this issue when I decided to leave it open in case someone wanted to propose something and another core dev wanted to take it on. But everything is working as I expect it to and you may want to do your own implementation on PyPI if you want fancier as it's already a delicate thing as it is and so adding complexity for this specific case is a tough sell.
Date User Action Args
2022-04-11 14:59:37adminsetgithub: 86439
2020-11-12 20:41:25brett.cannonsetnosy: - brett.cannon
2020-11-12 20:41:19brett.cannonsetmessages: + msg380848
2020-11-12 19:23:37epainesetnosy: + brett.cannon
messages: + msg380844
2020-11-10 22:09:01KevKeatingsetmessages: + msg380720
2020-11-10 22:06:51KevKeatingsetstatus: open

messages: + msg380719
2020-11-10 22:00:01brett.cannonsetstatus: closed -> (no value)
nosy: - brett.cannon
resolution: not a bug ->
2020-11-10 21:57:48brett.cannonsetstatus: open -> closed
nosy: brett.cannon, ncoghlan, eric.snow, KevKeating, epaine
messages: + msg380718

resolution: not a bug
stage: resolved
2020-11-10 19:02:03epainesetmessages: + msg380694
versions: + Python 3.10
2020-11-09 21:53:20KevKeatingsetmessages: + msg380618
2020-11-09 21:31:45KevKeatingsetmessages: + msg380617
2020-11-07 17:37:08epainesetnosy: + brett.cannon, ncoghlan, eric.snow, epaine
messages: + msg380513

components: + Interpreter Core, - Library (Lib)
type: behavior
2020-11-05 22:14:23KevKeatingcreate