classification
Title: Aliasing import of sub-{module,package} from the package raises AttributeError on import.
Type: behavior Stage:
Components: Interpreter Core Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, brett.cannon, eric.snow, ezio.melotti, franck, ncoghlan, serhiy.storchaka
Priority: normal Keywords:

Created on 2015-01-09 06:44 by franck, last changed 2017-07-01 06:08 by serhiy.storchaka.

Messages (6)
msg233719 - (view) Author: Franck Michea (franck) * Date: 2015-01-09 06:44
Hi, for those of you that prefer to read an example, you can read that commented demonstration of the bug[1].

Today I discovered what I think is a bug in the import system. Here is the basic setup:

We have three nested packages: foo -> bar -> baz. The bar package imports foo.bar.baz. We try to import foo.bar. This works well unless we try to alias the foo.bar.baz import in foo.bar with the "import ... as ..." syntax. In that case the file will be read and executed but when we get out of the import statement, then python throws:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "foo_alias_mod/bar/__init__.py", line 1, in <module>
    import foo_alias_mod.bar.baz as name_does_not_matter
AttributeError: 'module' object has no attribute 'bar'

This works whether baz is a package or a module actually. It does not matter if it's from the interpreter, or in a file, ... I've seen it trigger with 2.7.5, 2.7.9, 3.4.5, tip, so I guess this has been here for some time.

Please read the link below for a complete demo, and you can always download the tarball[2] to test yourself.

[1]: Commented demonstration: http://98810f8c06.net/wtf_python.html
[2]: Tarball for test: http://98810f8c06.net/wtf_python-demo.tar.bz2
msg233744 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2015-01-09 10:09
You can see the difference between the two cases in the bytecode:

>>> dis.dis("import x.y.z")
  1           0 LOAD_CONST               0 (0)
              3 LOAD_CONST               1 (None)
              6 IMPORT_NAME              0 (x.y.z)
              9 STORE_NAME               1 (x)
             12 LOAD_CONST               1 (None)
             15 RETURN_VALUE                         
>>> dis.dis("import x.y.z as bar")            
  1           0 LOAD_CONST               0 (0)
              3 LOAD_CONST               1 (None)
              6 IMPORT_NAME              0 (x.y.z)
              9 LOAD_ATTR                1 (y)
             12 LOAD_ATTR                2 (z)
             15 STORE_NAME               3 (bar)
             18 LOAD_CONST               1 (None)
             21 RETURN_VALUE

The aliased version needs to bind the innermost object immediately, so it fails, since "foo.bar" doesn't get set until *after* the import is finished.

The version without the alias succeeeds, as it doesn't attempt to eagerly access the attribute before it gets set by the interpreter.

To better handle a similar situation with eager attribute lookups during import, issue 17636 changed IMPORT_FROM to fall back to looking at sys.modules when a module attribute it is looking for is missing.

Brett, Eric, perhaps it would be worth changing the bytecode emitted in the "import x.y.z as name" case to match that for "from x.y import x as name"?

>>> dis.dis("from x.y import z as bar")       
  1           0 LOAD_CONST               0 (0)
              3 LOAD_CONST               1 (('z',))
              6 IMPORT_NAME              0 (x.y)
              9 IMPORT_FROM              1 (z)
             12 STORE_NAME               2 (bar)
             15 POP_TOP
             16 LOAD_CONST               2 (None)
             19 RETURN_VALUE
msg233768 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-01-09 16:51
I think I would need to see exactly how you want the bytecode to change and then think over any backwards-compatibility issues since this is doesn't come up very often.
msg233815 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2015-01-10 11:06
I'm suggesting we change this part of the bytecode emitted for "import x.y.z as bar":

              6 IMPORT_NAME              0 (x.y.z)
              9 LOAD_ATTR                1 (y)
             12 LOAD_ATTR                2 (z)
             15 STORE_NAME               3 (bar)

to instead emit:

              6 IMPORT_NAME              0 (x.y)
              9 IMPORT_FROM              1 (z)
             12 STORE_NAME               2 (bar)
             15 POP_TOP

The degenerate case of "import x as y" would be unchanged, only cases which currently emit LOAD_ATTR instructions would be modified.

I haven't looked at the practical details yet, but the key would be to use IMPORT_FROM to do the name resolution, rather than LOAD_ATTR, thus enabling the fallback to sys.modules for the eager lookup.
msg233823 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-01-10 20:45
That seems reasonable. I guess first step is a patch and then seeing if it passes the test suite.
msg297476 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-07-01 06:08
Nick's suggestion was implemented in issue30024.
History
Date User Action Args
2017-07-01 06:08:50serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg297476
2015-06-29 22:32:21martin.panterlinkissue24530 superseder
2015-01-11 02:05:06Arfreversetnosy: + Arfrever
2015-01-10 20:45:27brett.cannonsetmessages: + msg233823
2015-01-10 11:06:23ncoghlansetmessages: + msg233815
2015-01-09 16:51:26brett.cannonsetmessages: + msg233768
2015-01-09 10:09:58ncoghlansetmessages: + msg233744
2015-01-09 07:45:01ezio.melottisetnosy: + brett.cannon, ncoghlan, ezio.melotti, eric.snow
type: behavior
2015-01-09 06:44:42franckcreate