classification
Title: inconsistent ImportError message executing same import statement
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 2.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, eric.snow, ncoghlan, xiang.zhang
Priority: low Keywords:

Created on 2018-01-02 08:48 by xiang.zhang, last changed 2018-06-08 14:26 by xiang.zhang. This issue is now closed.

Messages (3)
msg309360 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2018-01-02 08:48
While debugging a problem in my job I find an odd case about import, it could be reduced to the following tracebacks:

>>> from keystone.assignment import schema
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/keystone/assignment/__init__.py", line 15, in <module>
    from keystone.assignment import controllers  # noqa
  File "/usr/lib/python2.7/site-packages/keystone/assignment/controllers.py", line 26, in <module>
    from keystone.common import controller
  File "/usr/lib/python2.7/site-packages/keystone/common/controller.py", line 24, in <module>
    from keystone.common import authorization
  File "/usr/lib/python2.7/site-packages/keystone/common/authorization.py", line 23, in <module>
    from keystone.models import token_model
  File "/usr/lib/python2.7/site-packages/keystone/models/token_model.py", line 15, in <module>
    from keystoneclient.common import cms
ImportError: No module named keystoneclient.common
>>> from keystone.assignment import schema
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/keystone/assignment/__init__.py", line 15, in <module>
    from keystone.assignment import controllers  # noqa
  File "/usr/lib/python2.7/site-packages/keystone/assignment/controllers.py", line 25, in <module>
    from keystone.assignment import schema
ImportError: cannot import name schema

keystoneclient is deliberately not installed. I think it should always report keystoneclient.common could not be found which reveals the root causes. But only the first time it does so. Later tries it always report cannot import schema.

The reason of this behaviour is that first time although keystone.assignment import fails but keystone.assignment.schema is successfully installed in sys.modules. And then, in ensure_fromlist, it calls import_submodule which returns the module in sys.modules, without set schema attribute to keystone.assignment. So it then fails in import_from for not able to get the attribute.

It could be simply reproduced by the following package structure:

test
|-- a.py
|-- b.py
`-- __init__.py

__init__.py:
from test import a

a.py: 
from test import b
import modulenotexisting

b.py: #emtpy

>>> import test
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test/__init__.py", line 1, in <module>
    from test import a
  File "test/a.py", line 2, in <module>
    import dddddddddd
ImportError: No module named dddddddddd
>>> import test
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test/__init__.py", line 1, in <module>
    from test import a
  File "test/a.py", line 1, in <module>
    from test import b
ImportError: cannot import name b

Python3 doesn't suffer this problem, at least for 3.7.

I don't know this should be identified as a bug or not. But this does bring me trouble debugging problems and lead to confusions because it doesn't  tell users the real module can't be imported.
msg309381 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-01-02 15:59
Since the problem is specific to Python 2.7 and has been resolved in the importlib based import implementation, I'm inclined to close this as "Won't Fix".

The only reason I haven't is that if someone really wanted to dig into the Python 2.7 import implementation and figure out how to improve the situation, I'd actually be willing to review the patch.

Alternatively, it might be worthwhile for OpenStack to investigate migrating over to importlib2 for their Python 2.7 support (or at least supporting it as a debugging option).
msg309397 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2018-01-03 01:42
It's fine. Let's keep this issue open for some time before close it.
History
Date User Action Args
2018-06-08 14:26:04xiang.zhangsetstatus: open -> closed
resolution: wont fix
stage: resolved
2018-01-03 01:42:53xiang.zhangsetmessages: + msg309397
2018-01-02 15:59:17ncoghlansetpriority: normal -> low

messages: + msg309381
2018-01-02 08:48:08xiang.zhangcreate