This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients vstinner
Date 2019-06-13.09:36:05
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1560418565.72.0.288239825756.issue37266@roundup.psfhosted.org>
In-reply-to
Content
Py_EndInterpreter() calls threading._shutdown() which waits for non-daemon threads spawned in the subinterpreters. Problem: daemon threads continue to run after threading._shutdown(), but they rely on an interpreter which is being finalized and then deleted.

Attached example shows the problem:

$ ./python subinterp_daemon_thread.py 
hello from daemon thread
Fatal Python error: Py_EndInterpreter: not the last thread

Current thread 0x00007f13e5926740 (most recent call first):
  File "subinterp_daemon_thread.py", line 23 in <module>
Aborted (core dumped)

Catching the bug in Py_EndInterpreter() is too late. IMHO we must simply deny daemon threads by design in subinterpreters for safety.

In the main interpreter, we provide best effort to prevent crash at exit, but IMHO the implementation is ugly :-( ceval.c uses exit_thread_if_finalizing(): it immediately exit the current daemon thread if the threads attempts to acquire or release the GIL, whereas the interpreter is gone. Problem: we cannot release/clear some data structure at Python exit because of that. So Py_Finalize() may leak some memory by design, because of daemon threads.

IMHO we can be way stricter in subinterpreters.

I suggest to only modify Python 3.9.
History
Date User Action Args
2019-06-13 09:36:05vstinnersetrecipients: + vstinner
2019-06-13 09:36:05vstinnersetmessageid: <1560418565.72.0.288239825756.issue37266@roundup.psfhosted.org>
2019-06-13 09:36:05vstinnerlinkissue37266 messages
2019-06-13 09:36:05vstinnercreate