classification
Title: Move pending calls from _PyRuntime to PyInterpreterState
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: eric.snow, phsilva, shihai1991, vstinner
Priority: normal Keywords: patch

Created on 2020-03-17 00:53 by vstinner, last changed 2020-03-27 17:23 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19034 closed vstinner, 2020-03-17 01:01
PR 19047 merged vstinner, 2020-03-17 17:33
PR 19049 merged vstinner, 2020-03-17 23:51
PR 19050 merged vstinner, 2020-03-18 00:00
PR 19051 merged vstinner, 2020-03-18 00:19
PR 19054 merged vstinner, 2020-03-18 01:42
PR 19061 merged vstinner, 2020-03-18 18:03
PR 19066 merged vstinner, 2020-03-19 00:10
Messages (12)
msg364378 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-17 00:53
The _PyRuntime.ceval structure should be made "per-interpreter". I don't want to make the GIL per-interpreter: that's out of the scope of this issue. So I propose to only move a few fields to make more ceval fields "per interpreter".
msg364467 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-17 17:56
New changeset dab8423d220243efabbbcafafc12d90145539b50 by Victor Stinner in branch 'master':
bpo-39984: Add PyInterpreterState.ceval (GH-19047)
https://github.com/python/cpython/commit/dab8423d220243efabbbcafafc12d90145539b50
msg364491 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-18 00:50
While trying to move pending to PyInterpreterState, I hit in issue in _PyEval_AddPendingCall(): signal.raise_signal() can call it with tstate=NULL.

See https://bugs.python.org/issue37127#msg364489
msg364492 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-18 00:56
New changeset d7fabc116269e4650a684eb04f9ecd84421aa247 by Victor Stinner in branch 'master':
bpo-39984: Pass tstate to handle_signals() (GH-19050)
https://github.com/python/cpython/commit/d7fabc116269e4650a684eb04f9ecd84421aa247
msg364493 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-18 01:26
New changeset 23ef89db7ae46d160650263cc80479c2ed6693fb by Victor Stinner in branch 'master':
bpo-39984: _PyThreadState_DeleteCurrent() takes tstate (GH-19051)
https://github.com/python/cpython/commit/23ef89db7ae46d160650263cc80479c2ed6693fb
msg364495 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-18 02:04
New changeset 29356e03d4f8800b04f799efe7a10e3ce8b16f61 by Victor Stinner in branch 'master':
bpo-39877: Fix take_gil() for daemon threads (GH-19054)
https://github.com/python/cpython/commit/29356e03d4f8800b04f799efe7a10e3ce8b16f61
msg364503 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-18 08:26
New changeset 56bfdebfb17ea9d3245b1f222e92b8e3b1ed6118 by Victor Stinner in branch 'master':
bpo-39984: Pass tstate to _PyEval_SignalAsyncExc() (GH-19049)
https://github.com/python/cpython/commit/56bfdebfb17ea9d3245b1f222e92b8e3b1ed6118
msg364549 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-18 18:28
New changeset 8849e5962ba481d5d414b3467a256aba2134b4da by Victor Stinner in branch 'master':
bpo-39984: trip_signal() uses PyGILState_GetThisThreadState() (GH-19061)
https://github.com/python/cpython/commit/8849e5962ba481d5d414b3467a256aba2134b4da
msg364585 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-19 01:41
New changeset 50e6e991781db761c496561a995541ca8d83ff87 by Victor Stinner in branch 'master':
bpo-39984: Move pending calls to PyInterpreterState (GH-19066)
https://github.com/python/cpython/commit/50e6e991781db761c496561a995541ca8d83ff87
msg365006 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-25 17:57
I moved pending calls to PyInterpreterState.

There are remaining issues with using pending calls in subinterpreters, but I propose to continue the discussion in bpo-37127.
msg365161 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2020-03-27 16:21
Awesome!  Thanks for doing this, Victor.  I'll take a look when I can and adjust the changes for bpo-33608.  If you'll recall, I made a similar change as part of the solution for that issue, which we later reverted due to problems we discovered with daemon threads during runtime finalization.
msg365177 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-27 17:23
> Awesome!  Thanks for doing this, Victor.  I'll take a look when I can and adjust the changes for bpo-33608.  If you'll recall, I made a similar change as part of the solution for that issue, which we later reverted due to problems we discovered with daemon threads during runtime finalization.

I recall the revert dance, yeah. It took one year to fix the daemon threads and related issues! So far, buildbots look happy. I also ran many manual tests which makes me confident that the code is now reliable even with all recent changes done for isolating thread states.
History
Date User Action Args
2020-03-27 17:23:19vstinnersetmessages: + msg365177
2020-03-27 16:21:32eric.snowsetnosy: + eric.snow
messages: + msg365161
2020-03-25 17:57:41vstinnersetstatus: open -> closed
title: Move some ceval fields from _PyRuntime.ceval to PyInterpreterState.ceval -> Move pending calls from _PyRuntime to PyInterpreterState
messages: + msg365006

resolution: fixed
stage: patch review -> resolved
2020-03-19 04:34:07shihai1991setnosy: + shihai1991
2020-03-19 01:41:27vstinnersetmessages: + msg364585
2020-03-19 00:10:55vstinnersetpull_requests: + pull_request18420
2020-03-18 18:28:57vstinnersetmessages: + msg364549
2020-03-18 18:03:56vstinnersetpull_requests: + pull_request18414
2020-03-18 08:26:28vstinnersetmessages: + msg364503
2020-03-18 02:04:39vstinnersetmessages: + msg364495
2020-03-18 01:42:59vstinnersetpull_requests: + pull_request18406
2020-03-18 01:26:11vstinnersetmessages: + msg364493
2020-03-18 00:56:24vstinnersetmessages: + msg364492
2020-03-18 00:50:47vstinnersetmessages: + msg364491
2020-03-18 00:19:17vstinnersetpull_requests: + pull_request18402
2020-03-18 00:00:44vstinnersetpull_requests: + pull_request18401
2020-03-17 23:51:40vstinnersetpull_requests: + pull_request18400
2020-03-17 17:56:49vstinnersetmessages: + msg364467
2020-03-17 17:33:52vstinnersetpull_requests: + pull_request18398
2020-03-17 03:31:49phsilvasetnosy: + phsilva
2020-03-17 01:01:59vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request18383
2020-03-17 00:53:21vstinnercreate