New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test.support.import_fresh_module fails to correctly block submodules when fresh is specified #84354
Comments
It seems that test.support.import_fresh_module gets tripped up with its module blocking when you attempt to get a fresh copy of a submodule of a module where you are also importing the module that you are trying to block (bit of a doozy of a sentence there...). So, for example, with the following configuration in mymodule/init.py: from .other import other
try:
from ._a import attr
except ImportError:
from ._b import attr (Assuming _a.attr = "A" and _b.attr = "B"), if you attempt to do: m = test.support.import_fresh_module("mymodule", fresh=("mymodule._other",), blocked=("mymodule._a")) Then you'll find that m.attr is pulled from _a.attr. Here's a small script to demonstrate: from test.support import import_fresh_module
import sys
def import_ab(fresh_other):
fresh = ("mymodule._other", ) if fresh_other else ()
mods_out = []
for to_block in "_b", "_a":
blocked = (f"mymodule.{to_block}",)
mods_out.append(import_fresh_module("mymodule",
fresh=fresh, blocked=blocked))
return mods_out
for fresh_other in [True, False]:
mymodule_a, mymodule_b = import_ab(fresh_other)
qualifier = "With" if fresh_other else "Without"
print(f"{qualifier} a fresh import of mymodule._other")
print(f"a: {mymodule_a.attr}")
print(f"b: {mymodule_b.attr}")
print() When you run it with a suitably configured module on Python 3.8: $ python importer.py
With a fresh import of mymodule._other
a: A
b: A Without a fresh import of mymodule._other It also happens if you add I think the problem is that in the step where _save_and_remove_module is called on fresh_name (see here: cpython/Lib/test/support/__init__.py Lines 328 to 329 in 76db37b
sys.modules with a fresh import of the top-level module we're trying to import (mymodule) before the blocking goes into effect, then the final call to importlib.import_module just hits that cache.
I think either of the following options will fix this issue:
That said, I'm still having some weird statefulness problems if I block a C module's import and *then* block a Python module's import, so there may be some other underlying pathology to the current approach. |
Looks like deleting a module name after paul's example can be passed in PR19390. |
_save_and_remove_module in the old code did too much and too little. It tested that the fresh module is importable, saved the current state of the fresh module and its submodules and removed the fresh module and its submodules. Since it tested importability before saving all states, it can have a side effect of importing other fresh module, and since it did it before blocking, the imported "fresh" module could import the blocked modules.
It does not help, because the step of resolving "fresh" includes saving the sys.modules state, and the step of resolving "blocked" modifies sys.modules. Also the step of resolving "fresh" can modify sys.modules before saving its sate.
It would be not enough, because the problem can occur in one of additional fresh modules imported by the initial module. We need to remove all fresh modules before attempting to import them. The new import_fresh_module() consists of the following steps:
It does not save and restore the state of all modules, because some indirectly imported modules can not support repeated importing. It can be reconsidered in new versions. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: