classification
Title: Move pending calls from _PyRuntime to PyInterpreterState
Type: Stage: resolved
Components: Subinterpreters Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: eric.snow, phsilva, shihai1991, vstinner
Priority: normal Keywords: patch

Created on 2020-03-17 00:53 by vstinner, last changed 2020-05-15 00:37 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19034 closed vstinner, 2020-03-17 01:01
PR 19047 merged vstinner, 2020-03-17 17:33
PR 19049 merged vstinner, 2020-03-17 23:51
PR 19050 merged vstinner, 2020-03-18 00:00
PR 19051 merged vstinner, 2020-03-18 00:19
PR 19054 merged vstinner, 2020-03-18 01:42
PR 19061 merged vstinner, 2020-03-18 18:03
PR 19066 merged vstinner, 2020-03-19 00:10
Messages (14)
msg364378 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-17 00:53
The _PyRuntime.ceval structure should be made "per-interpreter". I don't want to make the GIL per-interpreter: that's out of the scope of this issue. So I propose to only move a few fields to make more ceval fields "per interpreter".
msg364467 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-17 17:56
New changeset dab8423d220243efabbbcafafc12d90145539b50 by Victor Stinner in branch 'master':
bpo-39984: Add PyInterpreterState.ceval (GH-19047)
https://github.com/python/cpython/commit/dab8423d220243efabbbcafafc12d90145539b50
msg364491 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-18 00:50
While trying to move pending to PyInterpreterState, I hit in issue in _PyEval_AddPendingCall(): signal.raise_signal() can call it with tstate=NULL.

See https://bugs.python.org/issue37127#msg364489
msg364492 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-18 00:56
New changeset d7fabc116269e4650a684eb04f9ecd84421aa247 by Victor Stinner in branch 'master':
bpo-39984: Pass tstate to handle_signals() (GH-19050)
https://github.com/python/cpython/commit/d7fabc116269e4650a684eb04f9ecd84421aa247
msg364493 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-18 01:26
New changeset 23ef89db7ae46d160650263cc80479c2ed6693fb by Victor Stinner in branch 'master':
bpo-39984: _PyThreadState_DeleteCurrent() takes tstate (GH-19051)
https://github.com/python/cpython/commit/23ef89db7ae46d160650263cc80479c2ed6693fb
msg364495 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-18 02:04
New changeset 29356e03d4f8800b04f799efe7a10e3ce8b16f61 by Victor Stinner in branch 'master':
bpo-39877: Fix take_gil() for daemon threads (GH-19054)
https://github.com/python/cpython/commit/29356e03d4f8800b04f799efe7a10e3ce8b16f61
msg364503 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-18 08:26
New changeset 56bfdebfb17ea9d3245b1f222e92b8e3b1ed6118 by Victor Stinner in branch 'master':
bpo-39984: Pass tstate to _PyEval_SignalAsyncExc() (GH-19049)
https://github.com/python/cpython/commit/56bfdebfb17ea9d3245b1f222e92b8e3b1ed6118
msg364549 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-18 18:28
New changeset 8849e5962ba481d5d414b3467a256aba2134b4da by Victor Stinner in branch 'master':
bpo-39984: trip_signal() uses PyGILState_GetThisThreadState() (GH-19061)
https://github.com/python/cpython/commit/8849e5962ba481d5d414b3467a256aba2134b4da
msg364585 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-19 01:41
New changeset 50e6e991781db761c496561a995541ca8d83ff87 by Victor Stinner in branch 'master':
bpo-39984: Move pending calls to PyInterpreterState (GH-19066)
https://github.com/python/cpython/commit/50e6e991781db761c496561a995541ca8d83ff87
msg365006 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-25 17:57
I moved pending calls to PyInterpreterState.

There are remaining issues with using pending calls in subinterpreters, but I propose to continue the discussion in bpo-37127.
msg365161 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2020-03-27 16:21
Awesome!  Thanks for doing this, Victor.  I'll take a look when I can and adjust the changes for bpo-33608.  If you'll recall, I made a similar change as part of the solution for that issue, which we later reverted due to problems we discovered with daemon threads during runtime finalization.
msg365177 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-27 17:23
> Awesome!  Thanks for doing this, Victor.  I'll take a look when I can and adjust the changes for bpo-33608.  If you'll recall, I made a similar change as part of the solution for that issue, which we later reverted due to problems we discovered with daemon threads during runtime finalization.

I recall the revert dance, yeah. It took one year to fix the daemon threads and related issues! So far, buildbots look happy. I also ran many manual tests which makes me confident that the code is now reliable even with all recent changes done for isolating thread states.
msg365998 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-08 17:44
I reopen the issue because of bpo-40082 "trip_signal() gets NULL tstate on Windows on CTRL+C" regression.
msg366018 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-08 21:53
> I reopen the issue because of bpo-40082 "trip_signal() gets NULL tstate on Windows on CTRL+C" regression.

bpo-40082 is fixed, so I close the issue again.
History
Date User Action Args
2020-05-15 00:37:52vstinnersetcomponents: + Subinterpreters, - Interpreter Core
2020-04-08 21:53:48vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg366018
2020-04-08 17:44:39vstinnersetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg365998
2020-03-27 17:23:19vstinnersetmessages: + msg365177
2020-03-27 16:21:32eric.snowsetnosy: + eric.snow
messages: + msg365161
2020-03-25 17:57:41vstinnersetstatus: open -> closed
title: Move some ceval fields from _PyRuntime.ceval to PyInterpreterState.ceval -> Move pending calls from _PyRuntime to PyInterpreterState
messages: + msg365006

resolution: fixed
stage: patch review -> resolved
2020-03-19 04:34:07shihai1991setnosy: + shihai1991
2020-03-19 01:41:27vstinnersetmessages: + msg364585
2020-03-19 00:10:55vstinnersetpull_requests: + pull_request18420
2020-03-18 18:28:57vstinnersetmessages: + msg364549
2020-03-18 18:03:56vstinnersetpull_requests: + pull_request18414
2020-03-18 08:26:28vstinnersetmessages: + msg364503
2020-03-18 02:04:39vstinnersetmessages: + msg364495
2020-03-18 01:42:59vstinnersetpull_requests: + pull_request18406
2020-03-18 01:26:11vstinnersetmessages: + msg364493
2020-03-18 00:56:24vstinnersetmessages: + msg364492
2020-03-18 00:50:47vstinnersetmessages: + msg364491
2020-03-18 00:19:17vstinnersetpull_requests: + pull_request18402
2020-03-18 00:00:44vstinnersetpull_requests: + pull_request18401
2020-03-17 23:51:40vstinnersetpull_requests: + pull_request18400
2020-03-17 17:56:49vstinnersetmessages: + msg364467
2020-03-17 17:33:52vstinnersetpull_requests: + pull_request18398
2020-03-17 03:31:49phsilvasetnosy: + phsilva
2020-03-17 01:01:59vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request18383
2020-03-17 00:53:21vstinnercreate