This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author p-ganssle
Recipients brett.cannon, eric.snow, ncoghlan, p-ganssle
Date 2020-04-03.15:18:43
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1585927123.84.0.0735169096885.issue40173@roundup.psfhosted.org>
In-reply-to
Content
It seems that test.support.import_fresh_module gets tripped up with its module blocking when you attempt to get a fresh copy of a submodule of a module where you are also importing the module that you are trying to block (bit of a doozy of a sentence there...). So, for example, with the following configuration in mymodule/__init__.py:

    from .other import other
    
    try:
        from ._a import attr
    except ImportError:
        from ._b import attr

(Assuming _a.attr = "A" and _b.attr = "B"), if you attempt to do:

    m = test.support.import_fresh_module("mymodule", fresh=("mymodule._other",), blocked=("mymodule._a"))

Then you'll find that m.attr is pulled from _a.attr. Here's a small script to demonstrate:

    from test.support import import_fresh_module
    import sys

    def import_ab(fresh_other):
        fresh = ("mymodule._other", ) if fresh_other else ()

        mods_out = []
        for to_block in "_b", "_a":
            blocked = (f"mymodule.{to_block}",)

            mods_out.append(import_fresh_module("mymodule",
                                                fresh=fresh, blocked=blocked))
        return mods_out


    for fresh_other in [True, False]:
        mymodule_a, mymodule_b = import_ab(fresh_other)

        qualifier = "With" if fresh_other else "Without"
        print(f"{qualifier} a fresh import of mymodule._other")

        print(f"a: {mymodule_a.attr}")
        print(f"b: {mymodule_b.attr}")
        print()

When you run it with a suitably configured module on Python 3.8:

$ python importer.py 
With a fresh import of mymodule._other
a: A
b: A

Without a fresh import of mymodule._other
a: A
b: B

It also happens if you add `mymodule._a` or `mymodule._b` to the fresh list when you are trying to block the other one.

I *think* the problem is that in the step where _save_and_remove_module is called on fresh_name (see here: https://github.com/python/cpython/blob/76db37b1d37a9daadd9e5b320f2d5a53cd1352ec/Lib/test/support/__init__.py#L328-L329), it's necessarily populating `sys.modules` with a fresh import of the top-level module we're trying to import (mymodule) *before* the blocking goes into effect, then the final call to importlib.import_module just hits that cache.

I think either of the following options will fix this issue:

1. Switching the order of how "fresh" and "blocked" are resolved or
2. Deleting `sys.modules[name]` if it exists immediately before calling `importlib.import_module(name)

That said, I'm still having some weird statefulness problems if I block a C module's import and *then* block a Python module's import, so there may be some other underlying pathology to the current approach.
History
Date User Action Args
2020-04-03 15:18:43p-gansslesetrecipients: + p-ganssle, brett.cannon, ncoghlan, eric.snow
2020-04-03 15:18:43p-gansslesetmessageid: <1585927123.84.0.0735169096885.issue40173@roundup.psfhosted.org>
2020-04-03 15:18:43p-gansslelinkissue40173 messages
2020-04-03 15:18:43p-gansslecreate