This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: [subinterpreters] Support basic asynchronous cross-interpreter operations.
Type: enhancement Stage: patch review
Components: C API, Subinterpreters Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: eric.snow Nosy List: eric.snow, vstinner
Priority: normal Keywords: patch

Created on 2020-05-08 22:28 by eric.snow, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 20012 open eric.snow, 2020-05-08 22:38
Messages (1)
msg368476 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2020-05-08 22:28
(This is a continuation of the work from bpo-33608.  That issue ended up with a lot of baggage and clutter (due to problems that have since been resolved), so we closed it.  This issue is where we're picking it up fresh.)

When two interpreters are cooperating, there are sometimes operations that one of them needs the other to perform.  Thus far this is limited to mutation or release of data/objects owned by that "other" interpreter.  We need safe, reliable public C-API to facilitate such operations.

All the necessary machinery already exists ("pending calls"), exposed in the internal C-API: _Py_AddPendingCall().  (There is a public Py_AddPendingCall(), but it should be removed.)  That API adds a function to a per-interpreter queue, from which it is executed later (asynchronously) by the ceval loop of the interpreter's main thread.  The challenge is that the repercussions of such async operations within the eval loop are not fully explored.  We have some confidence because this is the same machinery used to handle signals.  However, it's better avoid as much complexity in those async operations as possible.  That is why we don't just expose `_Py_AddPendingCall()` in the public C-API.

Instead the plan is to add a small number of public C-API functions that are each specific to a focused use case, ideally with with little/no chance of errors or other side effects.  Initially we will target `Py_DECREF()`, `PyMem_Free()`, and `PyBuffer_Release()`.  If we need more then we can assess those needs later.
History
Date User Action Args
2022-04-11 14:59:30adminsetgithub: 84752
2020-05-15 00:43:16vstinnersetcomponents: + Subinterpreters
title: Support basic asynchronous cross-interpreter operations. -> [subinterpreters] Support basic asynchronous cross-interpreter operations.
2020-05-08 22:38:02eric.snowsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request19323
2020-05-08 22:28:51eric.snowcreate