This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: importlib: better error message when import fail during Python shutdown
Type: Stage:
Components: Library (Lib) Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, eric.snow, python-dev, vstinner
Priority: normal Keywords: patch

Created on 2016-03-24 15:06 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
importlib_shutdown.patch vstinner, 2016-03-24 15:06 review
importlib_shutdown-2.patch vstinner, 2016-03-24 16:57 review
Messages (9)
msg262352 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-03-24 15:06
Example of script.py:
-------------------------
class Bla:
    def __del__(self):
        try:
            import xxxx
        except Exception as exc:
            print("import error: [%s] %r" % (type(exc), exc))

bla = Bla()
-------------------------

Running this example logs a strange error:
-------------------------
$ python3.5 script.py
import error: [<class 'TypeError'>] TypeError("'NoneType' object is not iterable",)
-------------------------

The error comes from importlib._bootstrap._find_spec() which tries to iterator on sys.meta_path, whereas PyImport_Cleanup() was called and this function setted sys.meta_path to None.

Attached patch enhances _find_spec() to handle this case to return None. Error with the patch:
-------------------------
$ python3.5 script.py
import error: [<class 'ImportError'>] ImportError('sys.meta_path is None, Python is likely shutting down',)
-------------------------
msg262367 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-03-24 16:57
Updated patch with a dot in the comment :-)

Is ImportError the best exception in this case?
msg262368 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-03-24 17:00
Either ImportError or TypeError. ImportError makes sense since this is directly related to import semantics while TypeError makes sense as we are saying None specifically is not allowed.
msg262375 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-03-24 20:49
I will reply to my own question: in fact, ImportError is more convenient. The "try/except ImportError" pattern is common, in some cases, it's acceptable to skip a whole module. For a direct example on import during Python shutdown, see the issue #21925: "ResourceWarning sometimes doesn't display" which adds "try/except ImportError" to be able to log warnings.

@Brett, Eric: So, do you like importlib_shutdown-2.patch?
msg262376 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-03-24 20:49
Oh. And is it ok to apply this change to Python 2.7 and 3.5?
msg262378 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-03-24 21:56
It's technically a change in semantics as it shifts what exception is raised, so I wouldn't backport it.
msg262379 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-03-24 21:59
> It's technically a change in semantics as it shifts what exception is raised, so I wouldn't backport it.

Ok, I'm fine with only changing Python 3.6.
msg262387 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-03-24 23:41
New changeset 14c56f697a85 by Victor Stinner in branch 'default':
Fix bug in __import__ during Python shutdown
https://hg.python.org/cpython/rev/14c56f697a85
msg262405 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-03-25 07:57
New changeset e2c482b60776 by Victor Stinner in branch 'default':
Issue #26637: Fix test_io
https://hg.python.org/cpython/rev/e2c482b60776
History
Date User Action Args
2022-04-11 14:58:28adminsetgithub: 70824
2016-03-25 07:57:50python-devsetmessages: + msg262405
2016-03-24 23:42:05vstinnersetstatus: open -> closed
resolution: fixed
2016-03-24 23:41:57python-devsetnosy: + python-dev
messages: + msg262387
2016-03-24 21:59:26vstinnersetmessages: + msg262379
2016-03-24 21:56:42brett.cannonsetmessages: + msg262378
2016-03-24 20:49:29vstinnersetmessages: + msg262376
2016-03-24 20:49:08vstinnersetmessages: + msg262375
2016-03-24 17:00:11brett.cannonsetmessages: + msg262368
2016-03-24 16:57:45vstinnersetfiles: + importlib_shutdown-2.patch

messages: + msg262367
2016-03-24 15:06:33vstinnercreate