Author vstinner
Recipients Johan Dahlin, db3l, emilyemorehouse, eric.snow, nascheme, ncoghlan, pmpp, serhiy.storchaka, vstinner, yselivanov
Date 2019-03-05.11:23:11
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
> That's okay, Victor.  Thanks for jumping on this.  I'll take a look when I get a chance.

From what I saw, your first commit was enough to reproduce the crash.

If I recall correctly, Antoine Pitrou modified the GIL so threads exit immediately when Py_Finalize() is called. I'm thinking at:

PyEval_RestoreThread(PyThreadState *tstate)
    /* _Py_Finalizing is protected by the GIL */
    if (_Py_IsFinalizing() && !_Py_CURRENTLY_FINALIZING(tstate)) {

Problem: this code uses tstate, whereas the crash occurred because tstate pointed to freed memory:

Thread 1 got the crash

(gdb) p *tstate
$3 = {
  prev = 0xdbdbdbdbdbdbdbdb,
  next = 0xdbdbdbdbdbdbdbdb,
  interp = 0xdbdbdbdbdbdbdbdb,


Thread 1 (LWP 100696):
#0  0x0000000000368210 in take_gil (tstate=0x8027e2050) at Python/ceval_gil.h:216
#1  0x0000000000368a94 in PyEval_RestoreThread (tstate=0x8027e2050) at Python/ceval.c:281

When this crash occurred, Py_Finalize() already completed in the main thread!

void _Py_NO_RETURN
Py_Exit(int sts)
    if (Py_FinalizeEx() < 0) {  /* <==== DONE! */
        sts = 120;

    exit(sts);    /* <=============== Crash occurred here! */

Py_Finalize() is supposed to wait for threads before deleting Python thread states:


    /* The interpreter is still entirely intact at this point, and the
     * exit funcs may be relying on that.  In particular, if some thread
     * or exit func is still waiting to do an import, the import machinery
     * expects Py_IsInitialized() to return true.  So don't say the
     * interpreter is uninitialized until after the exit funcs have run.
     * Note that uses an exit func to do a join on all the
     * threads created thru it, so this also protects pending imports in
     * the threads created via Threading.


    /* Remaining threads (e.g. daemon threads) will automatically exit
       after taking the GIL (in PyEval_RestoreThread()). */
    _PyRuntime.finalizing = tstate;
    _PyRuntime.initialized = 0;
    _PyRuntime.core_initialized = 0;

    /* Delete current thread. After this, many C API calls become crashy. */



The real problem for years are *deamon threads* which... BY DESIGN... remain alive after Py_Finalize() exit! But as I explained, they must exit as soon at they attempt to get GIL.
Date User Action Args
2019-03-05 11:23:11vstinnersetrecipients: + vstinner, nascheme, db3l, ncoghlan, pmpp, eric.snow, serhiy.storchaka, yselivanov, emilyemorehouse, Johan Dahlin
2019-03-05 11:23:11vstinnersetmessageid: <>
2019-03-05 11:23:11vstinnerlinkissue33608 messages
2019-03-05 11:23:11vstinnercreate