Author gvanrossum
Recipients Victor.Varvariuc, barry, brett.cannon, eric.snow, gvanrossum, irdb, ncoghlan
Date 2017-04-10.00:32:41
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1491784363.7.0.975896579055.issue30024@psf.upfronthosting.co.za>
In-reply-to
Content
So to save others the bother to check what's in the zip file, the contents of mytest/mod1.py is as follows:

import mytest.mod2 as mod


def func():
    print('mod1.func called')
    mod.func()

There's no __init__.py (so mytest is a namespace package, PEP 420) but adding an empty __init__.py makes no difference. The problem occurs with both Python 2 and 3.

The root cause is the import cycle. I played around with dis and found the following (I suspect others have already found this but the thread was hard to follow for me initially):

>>> dis.dis('import a.b')
  1           0 LOAD_CONST               0 (0)
              2 LOAD_CONST               1 (None)
              4 IMPORT_NAME              0 (a.b)
              6 STORE_NAME               1 (a)
              8 LOAD_CONST               1 (None)
             10 RETURN_VALUE
>>>

compared to

>>> dis.dis('import a.b as c')
  1           0 LOAD_CONST               0 (0)
              2 LOAD_CONST               1 (None)
              4 IMPORT_NAME              0 (a.b)
              6 LOAD_ATTR                1 (b)      <-- error here
              8 STORE_NAME               2 (c)
             10 LOAD_CONST               1 (None)
             12 RETURN_VALUE
>>>

What this shows is that the implementation of "import a.b" and "import a.b as c" are different. The former calls __import__('a.b', ...) which returns the module 'a' and stores that in the variable 'a'. In the OP's case, because of the import cycle, while sys.modules['a.b'] exists, module 'a' does not yet have the attribute 'b'. That's the reason that in the latter example, the LOAD_ATTR opcode fails.

I'm inclined not to attempt to fix this. The reason that we don't pull 'a.b' out of sys.modules at this point is that if later in the execution of a/b.py we get an exception, the import will be cancelled, and sys.modules['a.b'] will be *deleted*, and then the variable 'c' would have a reference to a half-loaded module.

The semantics of imports in the case of cycles are somewhat complex but clearly defined and there are only a few rules to consider, and from these rules it is possible to reason out whether any particular case is valid or not. I would prefer to keep it this way rather than add more special cases. There's a good reason why, *in general* (regardless of import cycles), "import a.b as c" is implemented as a getattr operation on a, not as an index operation on sys.modules (it is possible for module a to override its attribute b without updating sys.modules) and I'd rather keep those semantics than give them up for this particular edge case. Cyclic imports are hard. If they don't work for you, avoid them.
History
Date User Action Args
2017-04-10 00:32:43gvanrossumsetrecipients: + gvanrossum, barry, brett.cannon, ncoghlan, irdb, eric.snow, Victor.Varvariuc
2017-04-10 00:32:43gvanrossumsetmessageid: <1491784363.7.0.975896579055.issue30024@psf.upfronthosting.co.za>
2017-04-10 00:32:43gvanrossumlinkissue30024 messages
2017-04-10 00:32:41gvanrossumcreate