This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Trashcan mechanism segfault during interpreter finalization in Python 3.7.5
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: ronaldoussoren, ysnt27
Priority: normal Keywords:

Created on 2019-10-28 02:44 by ysnt27, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
sample1.cc ysnt27, 2019-10-28 02:44
Messages (4)
msg355510 - (view) Author: (ysnt27) Date: 2019-10-28 02:44
My original issue was reported to Boost.Python,
https://github.com/boostorg/python/issues/248
And I found similar issue
https://bugs.python.org/issue17703

It seems root cause exists in Python not Boost.Python.
As the issue #17703 is already closed, I want to open a new issue.

I attached a sample source code, "sample1.cc".
With the code, I can reproduce segmantation fault, like this.

% gdb /usr/local/python375/bin/python3.7dm
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-115.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/local/python375/bin/python3.7dm...done.
(gdb) run
Starting program: /usr/local/python375/bin/python3.7dm 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Python 3.7.5 (default, Oct 25 2019, 16:11:10) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sample1
something called
>>> sample1.dummy()
[1]
>>> exit()
~something called

Program received signal SIGSEGV, Segmentation fault.
0x000000000044602a in list_dealloc (op=op@entry=0x7ffff7f4b360)
    at ../Objects/listobject.c:316
316	    Py_TRASHCAN_SAFE_BEGIN(op)
(gdb) p _PyRuntime.gilstate.tstate_current
$1 = {_value = 0}
(gdb) where
#0  0x000000000044602a in list_dealloc (op=op@entry=0x7ffff7f4b360)
    at ../Objects/listobject.c:316
#1  0x0000000000466452 in _Py_Dealloc (op=0x7ffff7f4b360)
    at ../Objects/object.c:1971
#2  0x00007ffff051cc28 in something::~something (this=0x7ffff071e140 <obj>, 
    __in_chrg=<optimized out>) at sample1.cc:26
#3  0x00007ffff6eeac99 in __run_exit_handlers (status=status@entry=0, 
    listp=0x7ffff72786c8 <__exit_funcs>, 
    run_list_atexit=run_list_atexit@entry=true) at exit.c:77
#4  0x00007ffff6eeace7 in __GI_exit (status=status@entry=0) at exit.c:99
#5  0x00000000005178e4 in Py_Exit (sts=sts@entry=0)
    at ../Python/pylifecycle.c:2292
#6  0x000000000051faf4 in handle_system_exit () at ../Python/pythonrun.c:636
#7  0x0000000000520d88 in PyErr_PrintEx (
    set_sys_last_vars=set_sys_last_vars@entry=1) at ../Python/pythonrun.c:646
#8  0x000000000052115f in PyErr_Print () at ../Python/pythonrun.c:542
#9  0x0000000000521d1a in PyRun_InteractiveLoopFlags (
    fp=fp@entry=0x7ffff7279640 <_IO_2_1_stdin_>, 
    filename_str=filename_str@entry=0x5e55b2 "<stdin>", 
    flags=flags@entry=0x7fffffffe2f0) at ../Python/pythonrun.c:134
#10 0x00000000005223d8 in PyRun_AnyFileExFlags (
    fp=fp@entry=0x7ffff7279640 <_IO_2_1_stdin_>, filename=0x5e55b2 "<stdin>", 
    closeit=closeit@entry=0, flags=flags@entry=0x7fffffffe2f0)
---Type <return> to continue, or q <return> to quit---
    at ../Python/pythonrun.c:78
#11 0x0000000000422023 in pymain_run_file (fp=0x7ffff7279640 <_IO_2_1_stdin_>, 
    filename=0x0, p_cf=p_cf@entry=0x7fffffffe2f0) at ../Modules/main.c:427
#12 0x00000000004220e5 in pymain_run_filename (
    pymain=pymain@entry=0x7fffffffe320, cf=cf@entry=0x7fffffffe2f0)
    at ../Modules/main.c:1606
#13 0x00000000004221d8 in pymain_run_python (pymain=pymain@entry=0x7fffffffe320)
    at ../Modules/main.c:2867
#14 0x000000000042550f in pymain_main (pymain=pymain@entry=0x7fffffffe320)
    at ../Modules/main.c:3028
#15 0x0000000000425589 in _Py_UnixMain (argc=<optimized out>, 
    argv=<optimized out>) at ../Modules/main.c:3063
#16 0x000000000041f116 in main (argc=<optimized out>, argv=<optimized out>)
    at ../Programs/python.c:15
(gdb) l
311	static void
312	list_dealloc(PyListObject *op)
313	{
314	    Py_ssize_t i;
315	    PyObject_GC_UnTrack(op);
316	    Py_TRASHCAN_SAFE_BEGIN(op)
317	    if (op->ob_item != NULL) {
318	        /* Do it backwards, for Christian Tismer.
319	           There's a simple test case where somehow this reduces
320	           thrashing when a *very* large list is created and
(gdb)
msg356062 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2019-11-05 21:05
The attached code creates a static (global) C++ object that owns a reference to a Python object, and releases that reference in its destructor. That destructor runs at program termination, which is after interpreter shutdown (that is after Py_FinalizeEx is called).  

After the call to Py_FinalizeEx() the interpreter no longer exists, and it is unsafe to call Python API functions (other than the ones listed as safe to call before a call to Py_Initialize).  Py_DECREF is not on that safe list.
msg356568 - (view) Author: (ysnt27) Date: 2019-11-14 03:07
Thanks for making the issue clear.

My understanding is that,
all C++ destructors have to check Python interpreter before Py_DECREF,
like this.

``` cplusplus
  ~something() {
    // PyThreadState *_tstate = PyThreadState_GET();
    PyThreadState *_tstate = _PyThreadState_UncheckedGet();
    if (_tstate) {
      Py_DECREF(somelist);
    }
  }
```

Is this right?
msg356585 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2019-11-14 07:55
You only have to check, using a public API, if you have no control over when C++ objects are destructed. I'd personally try to structure code to make sure that C++ objects owning references to Python objects get destructed before the interpreter is finalised (or release their references). 

The code you mention is not using a public API, Py_IsInitialized <https://docs.python.org/3/c-api/init.html#c.Py_IsInitialized> is a public API that can be used to check if the interpreter is initialised.
History
Date User Action Args
2022-04-11 14:59:22adminsetgithub: 82790
2019-11-14 07:55:33ronaldoussorensetstatus: open -> closed
resolution: not a bug
messages: + msg356585

stage: resolved
2019-11-14 03:07:48ysnt27setmessages: + msg356568
2019-11-05 21:05:54ronaldoussorensetnosy: + ronaldoussoren
messages: + msg356062
2019-10-28 02:44:52ysnt27create