classification
Title: Improve AttributeError message for partially initialized module
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, eric.snow, ncoghlan, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2018-04-06 15:43 by serhiy.storchaka, last changed 2018-10-30 11:21 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 6398 merged serhiy.storchaka, 2018-04-06 15:45
Messages (8)
msg315019 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-06 15:43
Cyclic import usually leads to an AttributeError "module 'spam' has no attribute 'ham'" which usually is confusing because in normal case 'spam.ham' exists, and the user can have no ideas why it is disappeared.

The proposed PR allows to specialize the AttributeError message for partially initialized module. Any suggestions about the error message?
msg315020 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-04-06 15:57
While I like the idea of this change, the "partially initialized" addition is fairly subtle, and relatively easy to miss.

Perhaps append "(most likely due to a circular import)" to the partially initialized case?:

    AttributeError: partially initialized "module 'spam' has no attribute 'ham' (most likely due to a circular import).

Crucially, for folks encountering the error for the first time, that also introduces them to the main phrase they may want to search for: "circular import".

The "most likely" weasel wording stems from the fact that the problem won't always be with the circular import - they may have just straight up referenced the wrong module or the wrong attribute name, so the apparently circular import is an error.
msg315071 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2018-04-07 19:01
+1 from me for Nick's suggestion.
msg315076 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-04-08 01:29
Oops, just realised my suggested text had an extraneous double quote in it due to a copy-and-paste error. Fixed version:

    AttributeError: partially initialized module 'spam' has no attribute 'ham' (most likely due to a circular import).
msg315136 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-04-09 16:26
I have applied the Nick's suggestion. Needed to find a place for new test.

The code is copied from PyImport_ImportModuleLevelObject(). I'm not happy from how verbose it is. And testing mod.__spec__._initialized adds relatively large overhead for importing already imported module in PyImport_ImportModuleLevelObject(). Is it possible to invent a faster way for checking whether the module is partially imported?
msg315166 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-04-10 11:28
The main idea that comes to mind is to cache a reference to `_frozen_importlib._module_locks` in the interpreter state, and do a key lookup in there (since any in-progress import should have a lock allocated to it).

That would be a separate performance issue though - for this issue, we're on an error handling path, so the speed with which the error gets reported isn't critical (although it does technically slow down try/except import fallback chains).
msg328131 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-10-20 07:52
Example:

$ cat foo.py
import bar
bar.baz
$ cat bar.py
import foo
baz = 2
$ ./python foo.py 
Traceback (most recent call last):
  File "foo.py", line 1, in <module>
    import bar
  File "/home/serhiy/py/cpython/bar.py", line 1, in <module>
    import foo
  File "/home/serhiy/py/cpython/foo.py", line 2, in <module>
    bar.baz
AttributeError: module 'bar' has no attribute 'baz'

Patched:

$ ./python foo.py 
Traceback (most recent call last):
  File "foo.py", line 1, in <module>
    import bar
  File "/home/serhiy/py/cpython/bar.py", line 1, in <module>
    import foo
  File "/home/serhiy/py/cpython/foo.py", line 2, in <module>
    bar.baz
AttributeError: partially initialized module 'bar' has no attribute 'baz' (most likely due to a circular import)
msg328895 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-10-30 11:19
New changeset 3e429dcc242e48fa4cbb1a91cf7c416c37b97b4e by Serhiy Storchaka in branch 'master':
bpo-33237: Improve AttributeError message for partially initialized module. (GH-6398)
https://github.com/python/cpython/commit/3e429dcc242e48fa4cbb1a91cf7c416c37b97b4e
History
Date User Action Args
2018-10-30 11:21:29serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-10-30 11:19:53serhiy.storchakasetmessages: + msg328895
2018-10-20 07:52:13serhiy.storchakasetmessages: + msg328131
2018-04-10 11:28:55ncoghlansetmessages: + msg315166
2018-04-09 16:26:29serhiy.storchakasetmessages: + msg315136
2018-04-08 01:29:57ncoghlansetmessages: + msg315076
2018-04-07 19:01:14brett.cannonsetmessages: + msg315071
2018-04-06 15:57:44ncoghlansetmessages: + msg315020
2018-04-06 15:45:24serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request6104
2018-04-06 15:43:32serhiy.storchakacreate