This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Deadlock between GIL and pystate head_mutex.
Type: Stage:
Components: Interpreter Core Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eric.snow, methane, pixelbeat
Priority: normal Keywords:

Created on 2016-11-27 10:13 by methane, last changed 2022-04-11 14:58 by admin.

Files
File name Uploaded Description Edit
finish-deadlock.py methane, 2016-11-27 10:13
Messages (3)
msg281816 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2016-11-27 10:13
While investigating #28673, I found another deadlock relating to finalization and threading.

deadlocked thread (holding gil / waiting head_mutex):
#0  0x00007f35dd400a07 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x5557e6d3eeb0) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  do_futex_wait (sem=sem@entry=0x5557e6d3eeb0, abstime=0x0) at sem_waitcommon.c:111
#2  0x00007f35dd400ab4 in __new_sem_wait_slow (sem=0x5557e6d3eeb0, abstime=0x0) at sem_waitcommon.c:181
#3  0x00007f35dd400b5a in __new_sem_wait (sem=sem@entry=0x5557e6d3eeb0) at sem_wait.c:29
#4  0x00005557e64ec01b in PyThread_acquire_lock_timed (lock=0x5557e6d3eeb0, microseconds=-1, intr_flag=intr_flag@entry=0) at Python/thread_pthread.h:352
#5  0x00005557e64ec148 in PyThread_acquire_lock (lock=<optimized out>, waitflag=waitflag@entry=1) at Python/thread_pthread.h:556
#6  0x00005557e64d8dbb in tstate_delete_common (tstate=tstate@entry=0x7f35cc002a80) at Python/pystate.c:426
#7  0x00005557e64d9e8d in PyThreadState_DeleteCurrent () at Python/pystate.c:462
#8  0x00005557e662d339 in t_bootstrap (boot_raw=0x7f35dc186628) at ./Modules/_threadmodule.c:1027
#9  0x00007f35dd3f870a in start_thread (arg=0x7f35c3fff700) at pthread_create.c:333
#10 0x00007f35dca220af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:105

main thread (waiting gil / holding head_mutex):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1  0x00005557e65f1758 in PyCOND_TIMEDWAIT (us=5000, mut=0x5557e69cc860 <gil_mutex>, cond=0x5557e69cc8a0 <gil_cond>) at Python/condvar.h:103
#2  take_gil (tstate=tstate@entry=0x5557e6d3f840) at Python/ceval_gil.h:224
#3  0x00005557e65f3b14 in _PyEval_EvalFrameDefault (f=Frame 0x7f35d0001878, for file finish-deadlock.py, line 7, in __del__ (self=<C at remote 0x7f35dc1b6878>), throwflag=<optimized out>)
    at Python/ceval.c:1140
#4  0x00005557e65f1dcf in PyEval_EvalFrameEx (f=f@entry=Frame 0x7f35d0001878, for file finish-deadlock.py, line 7, in __del__ (self=<C at remote 0x7f35dc1b6878>), throwflag=throwflag@entry=0)
    at Python/ceval.c:718
#5  0x00005557e65f1eab in _PyFunction_FastCall (co=co@entry=0x7f35dc50eac0, args=0x7fff1a1e2878, args@entry=0x7fff1a1e2870, nargs=nargs@entry=1, 
    globals=globals@entry={'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='finish-deadlock.py') at remote 0x7f35dd6c00c0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7f35dd791c58>, '__cached__': None, 'sys': <module at remote 0x7f35dd7b9858>, 'time': <module at remote 0x7f35dc4ea6d8>, 'threading': <module at remote 0x7f35dc4ea8d8>, 'C': <type at remote 0x5557e6dc96c8>, 'worker': <function at remote 0x7f35dc4d53f0>, '_': 4, 't': <Thread(_target=<function at remote 0x7f35dc4d53f0>, _name='Thread-11', _args=(), _kwargs={}, _daemonic=True, _ident=139869249451776, _tstate_lock=<_thread.lock at remote 0x7f35dc4739e0>, _started=<Event(_cond=<Condition(_lock=<_thread.lock at remote 0x7f35dc42be58>, acquire=<built-in method acquire of _thread.lock object at remote 0x7f35dc42be58>, release=<built-in method release of _thread.lock object at remote 0x7f35dc42be58>, _waiters=<collections.deque at remote 0x7f35dc16d5b0>) at remote 0x7f35dc191538>,...(truncated)) at Python/ceval.c:4881
#6  0x00005557e65feb04 in _PyFunction_FastCallDict (func=func@entry=<function at remote 0x7f35dc11f788>, args=args@entry=0x7fff1a1e2870, nargs=nargs@entry=1, kwargs=kwargs@entry=0x0)
    at Python/ceval.c:4983
#7  0x00005557e6500386 in _PyObject_FastCallDict (func=func@entry=<function at remote 0x7f35dc11f788>, args=args@entry=0x7fff1a1e2870, nargs=nargs@entry=1, kwargs=kwargs@entry=0x0)
    at Objects/abstract.c:2301
#8  0x00005557e6500644 in _PyObject_Call_Prepend (func=<function at remote 0x7f35dc11f788>, obj=obj@entry=<C at remote 0x7f35dc1b6878>, args=args@entry=(), kwargs=kwargs@entry=0x0)
    at Objects/abstract.c:2364
#9  0x00005557e651c2e4 in method_call (method=method@entry=<method at remote 0x7f35dc521758>, args=args@entry=(), kwargs=kwargs@entry=0x0) at Objects/classobject.c:317
#10 0x00005557e6500247 in _PyObject_FastCallDict (func=func@entry=<method at remote 0x7f35dc521758>, args=args@entry=0x0, nargs=nargs@entry=0, kwargs=kwargs@entry=0x0) at Objects/abstract.c:2322
#11 0x00005557e65f34a2 in PyEval_CallObjectWithKeywords (func=func@entry=<method at remote 0x7f35dc521758>, args=args@entry=0x0, kwargs=kwargs@entry=0x0) at Python/ceval.c:4705
#12 0x00005557e6577f0d in slot_tp_finalize (self=<optimized out>) at Objects/typeobject.c:6416
#13 0x00005557e655a07c in PyObject_CallFinalizer (self=self@entry=<C at remote 0x7f35dc1b6878>) at Objects/object.c:297
#14 0x00005557e655adb7 in PyObject_CallFinalizerFromDealloc (self=self@entry=<C at remote 0x7f35dc1b6878>) at Objects/object.c:314
#15 0x00005557e656cfde in subtype_dealloc (self=self@entry=<C at remote 0x7f35dc1b6878>) at Objects/typeobject.c:1151
#16 0x00005557e655ae75 in _Py_Dealloc (op=<C at remote 0x7f35dc1b6878>) at Objects/object.c:1786
#17 0x00005557e65312c5 in frame_dealloc (f=f@entry=Frame 0x7f35c40026a8, for file finish-deadlock.py, line 22, in worker ()) at Objects/frameobject.c:423
#18 0x00005557e655ae75 in _Py_Dealloc (op=Frame 0x7f35c40026a8, for file finish-deadlock.py, line 22, in worker ()) at Objects/object.c:1786
#19 0x00005557e64e88c1 in tb_dealloc (tb=tb@entry=0x7f35dc193f38) at Python/traceback.c:55
#20 0x00005557e655ae75 in _Py_Dealloc (op=<traceback at remote 0x7f35dc193f38>) at Objects/object.c:1786
#21 0x00005557e64d98f3 in PyThreadState_Clear (tstate=tstate@entry=0x5557e6e307f0) at Python/pystate.c:403
#22 0x00005557e64d99cf in PyInterpreterState_Clear (interp=interp@entry=0x5557e6d3f390) at Python/pystate.c:119
#23 0x00005557e64d7b0f in Py_FinalizeEx () at Python/pylifecycle.c:672
#24 0x00005557e64eed2f in Py_Main (argc=2, argv=0x5557e6d3e010) at Modules/main.c:800
#25 0x00005557e64d3ad5 in main (argc=2, argv=0x7fff1a1e2d78) at ./Programs/python.c:69
msg298128 - (view) Author: Pádraig Brady (pixelbeat) Date: 2017-07-11 06:20
This can probably closed as a dupe of #30395
msg298152 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2017-07-11 12:28
#30395 is about forking, but this is about multi threading.
History
Date User Action Args
2022-04-11 14:58:40adminsetgithub: 72998
2019-12-20 23:09:15eric.snowsetnosy: + eric.snow
2017-07-11 12:28:34methanesetmessages: + msg298152
2017-07-11 06:20:36pixelbeatsetnosy: + pixelbeat
messages: + msg298128
2016-11-27 10:13:18methanecreate