classification
Title: Surprising name binding behavior of submodule imports needs documenting
Type: behavior Stage:
Components: Documentation Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: barry Nosy List: barry, brett.cannon, eric.snow, python-dev
Priority: normal Keywords:

Created on 2015-04-22 17:14 by barry, last changed 2015-04-22 22:40 by barry. This issue is now closed.

Files
File name Uploaded Description Edit
issue24029-1.txt barry, 2015-04-22 21:49 review
Messages (8)
msg241816 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2015-04-22 17:14
As described here: http://news.gmane.org/find-root.php?message_id=20150422115959.1ff2ee58%40limelight.wooz.org

Importing a submodule binds the submodule's name in the parent module's namespace.  This is surprising, but it seems intentional and it's relied upon by existing code, e.g. asyncio/__init__.py in the stdlib.

It's also not documented afaict.  It should be documented in the Language Reference's section on the import system.  After a little more discussion on import-sig, I plan on doing that.
msg241817 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2015-04-22 17:20
From Guido:

It's definitely intentional, and it's fundamental to the package import
design. We've had many implementations of package import (remember "ni.py"?
last seen as "knee.py") and it was always there, because this is done as
part of *submodule loading*. For better or for worse (and because I didn't
know Java at the time :-) Python declares that if you write `import
foo.bar` then later in your code you can use `foo.bar` to reference to the
bar submodule of package foo. And the way this is done is to make each
submodule an attribute of its parent package. This is done when the
submodule is first loaded, and because of the strict separation between
loading and importing, it is done no matter what form of import was used to
load bar.

I guess another thing to realize is that the globals of __init__.py are
also the attribute namespace of the package.

I'm not surprised it's in the reference manual -- that hasn't been updated
thoroughly in ages, and I sometimes cry when I see it. :-) So please do
clarify this for the benefit of future implementers.
msg241818 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2015-04-22 17:50
More rationale from the thread:

> The surprising part is that it also happens for explicit relative
> imports.  I'm guessing that part was unintentional and simply not
> noticed when PEP 328 was implemented.
>  

No, that must also have been intentional, because even when you use
relative import, the module you imported knows its full name, and that full
name is used as its key in sys.modules. If someone else uses absolute
import for the same module they should still get the same module object.
msg241822 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-04-22 18:33
Guido describes the global invariant for *all* the forms of importing a submodule, including explicit relative imports:

> I just mean that for relative import
> there is no need to bind the submodule to the parent, is there?

But there *is* a reason. The submodule must still be an attribute of the parent package, because of the invariant that if you have sys.modules['foo'] and sys.modules['foo.bar'], the latter must appear as the 'bar' attribute of the former. This is an invariant of module loading, and (I feel I'm repeating myself) the form of import used does not affect loading.
msg241829 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2015-04-22 21:49
Here's some new text for the Language Reference.
msg241830 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-04-22 22:16
LGTM.  You've covered all the key points and the example is good.
msg241831 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2015-04-22 22:25
Cool, thanks!  I'll commit it and we can always clean it up/add to it later if needed.
msg241832 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-22 22:38
New changeset 3968e7a9614c by Barry Warsaw in branch '3.4':
Issue #24029: Document the name binding behavior for submodule imports.
https://hg.python.org/cpython/rev/3968e7a9614c

New changeset 351ad8c4f3a6 by Barry Warsaw in branch '3.4':
Issue #24029: Document the name binding behavior for submodule imports.
https://hg.python.org/cpython/rev/351ad8c4f3a6

New changeset 6295f207dfaa by Barry Warsaw in branch 'default':
Issue #24029: Document the name binding behavior for submodule imports.
https://hg.python.org/cpython/rev/6295f207dfaa
History
Date User Action Args
2015-04-22 22:40:30barrysetstatus: open -> closed
resolution: fixed
2015-04-22 22:38:37python-devsetnosy: + python-dev
messages: + msg241832
2015-04-22 22:25:55barrysetmessages: + msg241831
2015-04-22 22:16:27eric.snowsetmessages: + msg241830
2015-04-22 21:49:04barrysetfiles: + issue24029-1.txt

messages: + msg241829
2015-04-22 18:33:26eric.snowsetnosy: + eric.snow
messages: + msg241822
2015-04-22 17:50:52barrysetmessages: + msg241818
2015-04-22 17:20:15barrysetmessages: + msg241817
2015-04-22 17:14:50barrycreate