classification
Title: PyEval_AcquireLock() and PyEval_AcquireThread() do not handle runtime finalization properly.
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: eric.snow, nanjekyejoannah, vstinner
Priority: normal Keywords: patch

Created on 2019-03-29 19:26 by eric.snow, last changed 2019-05-04 15:48 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 12667 merged nanjekyejoannah, 2019-04-02 19:57
PR 13068 merged vstinner, 2019-05-02 19:35
Messages (6)
msg339138 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2019-03-29 19:26
Daemon threads keep running until they finish or until finalization starts.  For the latter, there is a check right after the thread acquires the GIL which causes the thread to exit if runtime finalization has started. [1]  However, there are functions in the C-API that facilitate acquiring the GIL, but do not cause the thread to exit during finalization:

  PyEval_AcquireLock()
  PyEval_AcquireThread()

Daemon threads that acquire the GIL through these can cause a deadlock during finalization.  (See issue #36469.)  They should probably be updated to match what PyEval_RestoreThread() does.


[1] see PyEval_RestoreThread() and the eval loop, in PyEval_EvalFrameEx()
msg339365 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python committer) Date: 2019-04-02 20:12
@eric.snow , you can review the PR I submitted for this.
msg339876 - (view) Author: Joannah Nanjekye (nanjekyejoannah) * (Python committer) Date: 2019-04-10 16:52
@eric do we need any tests for this?
msg341054 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-29 08:38
New changeset f781d202a2382731b43bade845a58d28a02e9ea1 by Victor Stinner (Joannah Nanjekye) in branch 'master':
bpo-36475: Finalize PyEval_AcquireLock() and PyEval_AcquireThread() properly (GH-12667)
https://github.com/python/cpython/commit/f781d202a2382731b43bade845a58d28a02e9ea1
msg341055 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-29 08:48
I am not comfortable to backport this change to Python 3.7. It's too early to know how it will impact applications and how many complains we will get :-) If someone really wants to backport this scary change to 3.7, I would suggest to wait for 1 month after Python 3.8.0 final release.

I close the issue.

See bpo-36479 for the follow-up.
msg341389 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-05-04 15:48
New changeset c664b342a47e4b4650706d07e3e40a295e3a4407 by Victor Stinner in branch 'master':
bpo-36475: Make PyThread_exit_thread with _Py_NO_RETURN (GH-13068)
https://github.com/python/cpython/commit/c664b342a47e4b4650706d07e3e40a295e3a4407
History
Date User Action Args
2019-05-04 15:48:07vstinnersetmessages: + msg341389
2019-05-02 19:35:54vstinnersetpull_requests: + pull_request12985
2019-04-29 09:23:39vstinnersetstatus: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.8, - Python 3.6
2019-04-29 08:48:32vstinnersetmessages: + msg341055
versions: + Python 3.6, - Python 3.7, Python 3.8
2019-04-29 08:38:51vstinnersetmessages: + msg341054
2019-04-24 14:08:13vstinnersetnosy: + vstinner
2019-04-10 16:52:48nanjekyejoannahsetmessages: + msg339876
2019-04-02 20:12:59nanjekyejoannahsetnosy: + nanjekyejoannah
messages: + msg339365
2019-04-02 19:57:32nanjekyejoannahsetkeywords: + patch
stage: test needed -> patch review
pull_requests: + pull_request12595
2019-03-29 19:57:42eric.snowsetcomponents: + Interpreter Core
2019-03-29 19:26:20eric.snowcreate