classification
Title: Python crashes when a warning is emitted during shutdown
Type: crash Stage: resolved
Components: Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, python-dev, vstinner
Priority: normal Keywords:

Created on 2013-10-29 23:31 by vstinner, last changed 2013-10-31 23:55 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
warn_shutdown.py vstinner, 2013-10-29 23:53
Messages (6)
msg201695 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-10-29 23:31
Because of a bug, warnings were not emitted during Python shutdown. I fixed the bug in issue #19424, and now Python crashs when a warning is emitted during shutdown.

See also issue #19421: "FileIO destructor imports indirectly the io module at exit".

The warnings module uses globals()['__name__'] to get the name of the current module. The bug was the during Python shutdown, the module is None and _PyUnicode_AsString(None) was raising an exception (in setup_context() of _warnings.c) and so the warning was not emitted. For example, FileIO destructor emits a warning when a unclosed file is destroyed. The destructor removes the exception raised the warning module.
msg201696 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-10-29 23:35
New changeset 1787277915e9 by Victor Stinner in branch 'default':
Issue #19442: Fix warnings emitted during Python shutdown
http://hg.python.org/cpython/rev/1787277915e9
msg201699 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-10-29 23:39
The changeset 1787277915e9 is closer to a workaround than a real fix:

+    if (module != Py_None) {
+        res = warn_explicit(category, message, filename, lineno, module, registry,
+                            NULL);
+    }
+    else {
+        /* FIXME: emitting warnings at exit does crash Python */
+        res = Py_None;
+        Py_INCREF(res);
+    }
msg201700 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-10-29 23:53
warn_shutdown.py: example of script emitting warning at shutdown. Depending on the timing, the warning is shown or not (replace "if 0:" with "if 1:" to always display the warning). This example is a simplified example of the original crasher: test_threading.test_4_daemon_threads().
msg201861 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-10-31 23:53
warn_shutdown.py does not emit warnings because the thread states are destroyed too late, when the modules have been destroyed.

I opened a new issue #19466 "Clear state of threads earlier in Python shutdown" which would permit to emit properly warnings, at least for the specific case of warn_shutdown.py.

> The changeset 1787277915e9 is closer to a workaround than a real fix: ...

If the module is None, it's too late: modules have been destroyed, so it's safer to not call warn_explicit() and do nothing.

I tested to fail with an assertion error if a warning is emitted too late (ie. when module is None). test_threading.test_4_daemon_threads() does fail in this case. Using the patch attached to #19466, the full test suite pass correctly.
msg201862 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-10-31 23:55
New changeset 13a05ed33cf7 by Victor Stinner in branch 'default':
Close #19442: warn_explicit() does nothing when called late during Python shutdown
http://hg.python.org/cpython/rev/13a05ed33cf7
History
Date User Action Args
2013-10-31 23:55:45python-devsetstatus: open -> closed
resolution: fixed
messages: + msg201862

stage: resolved
2013-10-31 23:53:46vstinnersetmessages: + msg201861
2013-10-29 23:53:51vstinnersetfiles: + warn_shutdown.py

messages: + msg201700
2013-10-29 23:39:54vstinnersetmessages: + msg201699
2013-10-29 23:35:47python-devsetnosy: + python-dev
messages: + msg201696
2013-10-29 23:33:10Arfreversetnosy: + Arfrever

title: Python crashs when a warning is emitted during shutdown -> Python crashes when a warning is emitted during shutdown
2013-10-29 23:31:53vstinnercreate