classification
Title: Crash in _PyThreadState_DeleteExcept() at fork in the process child
Type: Stage:
Components: Interpreter Core Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: shihai1991, vstinner
Priority: normal Keywords:

Created on 2020-03-27 17:16 by vstinner, last changed 2020-04-28 05:12 by shihai1991.

Messages (5)
msg365173 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-27 17:16
At fork, Python calls PyOS_AfterFork_Child() in the child process which indirectly calls _PyThreadState_DeleteExcept() whichs calls release_sentinel() of the thread which releases the thread state lock (threading.Thread._tstate_lock).

Problem: using a lock after fork is unsafe and can crash.

That's exactly what happens randomly on AIX when stressing ThreadJoinOnShutdown.test_reinit_tls_after_fork() of test_threading:
https://bugs.python.org/issue40068#msg365031

There are different options to solve this issue:

* Reset _tstate_lock before using it... not sure that it's worth it, since we are going to delete the threading.Thread object with its _tstate_lock object anymore. After calling fork, the child process has exactly 1 thread: all other threads have been removed.

* Modify release_sentinel() to not use the lock: avoid PyThread_release_lock() call.
msg365175 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-27 17:18
See also bpo-40089: "Add _at_fork_reinit() method to locks".
msg367277 - (view) Author: hai shi (shihai1991) * (Python triager) Date: 2020-04-25 17:54
>since we are going to delete the threading.Thread object with its _tstate_lock object anymore.

Do we have any pep or discuss record about this plan?

> * Modify release_sentinel() to not use the lock: avoid PyThread_release_lock() call.

Hm, I am not sure about it. Looks like we can not remove it directly.
msg367422 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-27 13:42
> Do we have any pep or discuss record about this plan?

"since we are going to delete the threading.Thread object with its _tstate_lock object anyway" sentence is a description of the current _PyThreadState_DeleteExcept() implementation.
msg367496 - (view) Author: hai shi (shihai1991) * (Python triager) Date: 2020-04-28 05:12
> "since we are going to delete the threading.Thread object with its _tstate_lock object anyway" sentence is a description of the current _PyThreadState_DeleteExcept() implementation.

Oh, got it. thanks for your explanation :)
History
Date User Action Args
2020-04-28 05:12:33shihai1991setmessages: + msg367496
2020-04-27 13:42:26vstinnersetmessages: + msg367422
2020-04-25 17:54:07shihai1991setmessages: + msg367277
2020-04-17 17:13:22shihai1991setnosy: + shihai1991
2020-03-27 17:18:10vstinnersetmessages: + msg365175
2020-03-27 17:16:55vstinnercreate