This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Inconsistent import behavior for (unusual) submodules
Type: behavior Stage: needs patch
Components: Interpreter Core Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: eric.snow Nosy List: barry, brett.cannon, eric.snow, ncoghlan
Priority: normal Keywords:

Created on 2022-01-12 22:04 by eric.snow, last changed 2022-04-11 14:59 by admin.

Messages (3)
msg410433 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2022-01-12 22:04
Let's look at a hypothetical module "spam" and its submodule "spam.eggs":

```
# spam.py
import sys
sys.modules['spam.eggs'] = None
```

Then:

>>> import spam.eggs
>>> import sys
>>> sys.modules['spam.eggs'] is None
True
>>> spam.eggs
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'spam' has no attribute 'eggs'

The key inconsistent behaviors:

* `import spam.eggs` succeeded even though the sys.modules entry is None
* `import spam.eggs` succeeded even though "spam" isn't a package (e.g. no `__path__`, `spec.submodule_search_locations`, etc.)
* the "eggs" attr wasn't bound on "spam"

The relevant code is _find_and_load_unlocked() and _find_and_load() in Lib/importlib/_bootstrap.py.

In _find_and_load_unlocked() we first import the parent module.  Then we have a special case, where we see if "spam.eggs" was added to sys.modules as a side effect.  If it was then we short-circuit the rest of import and return the submodule as-is.  This leads to some of the inconsistent behavior described above, since the subsequent code (e.g. checks, binding to the parent) get skipped.

In _find_and_load() we have code which raises ModuleNotFoundError if the resulting module is None, which acts as a marker that importing the module is disabled.  This check is always skipped when importing the module for the first time, leading to the other inconsistent behavior from above.

The is definitely a corner case, but os.path demonstrates it's a real scenario.  In fact, os.path is what drew my attention to this code.

Is it worth fixing?  The change shouldn't be invasive so I'm leaning toward yes.  It isn't a high priority though.
msg410516 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2022-01-13 19:58
So which inconsistency do you want to change because you listed three and this is only one issue. 😉

I'm going to assume the "even though sys.modules has `None`" case, which I think is an oversight and should probably get fixed, but I also don't know what promises the language spec makes around this.

As for the other two, you can open separate issues if you want to discuss them, but I double-check what the language spec says as I am tempted to say both are fine (and specifically in the latter case that's on you to have not messed up and left the attribute off).
msg410524 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2022-01-13 21:08
> I'm going to assume the "even though sys.modules has `None`" case,
> which I think is an oversight and should probably get fixed

Yep, I agree.  That's the case I was looking at in the first place.  I noticed the other two as I was hacking together code to verify the None behavior. :)  Bothering to change those would be more trouble than its worth.
History
Date User Action Args
2022-04-11 14:59:54adminsetgithub: 90518
2022-01-13 21:08:31eric.snowsetmessages: + msg410524
2022-01-13 19:58:03brett.cannonsetmessages: + msg410516
2022-01-12 22:04:50eric.snowcreate