classification
Title: [subinterpreters][C API] Add a new function to create PyStructSequence from Heap.
Type: Stage: patch review
Components: C API Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: erlendaasland, petr.viktorin, rhettinger, shihai1991
Priority: normal Keywords: patch

Created on 2021-09-06 11:10 by shihai1991, last changed 2021-09-26 17:35 by shihai1991.

Pull Requests
URL Status Linked Edit
PR 28573 open shihai1991, 2021-09-26 17:35
Messages (6)
msg401129 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2021-09-06 11:10
Copied from https://bugs.python.org/issue40512#msg399847:

Victor: PyStructSequence_InitType2() is not compatible with subinterpreters: it uses static types. Moreover, it allocates tp_members memory which is not released when the type is destroyed. But I'm not sure that the type is ever destroyed, since this API is designed for static types.

> PyStructSequence_InitType2() is not compatible with subinterpreters: it uses static types. Moreover, it allocates tp_members memory which is not released when the type is destroyed. But I'm not sure that the type is ever destroyed, since this API is designed for static types.

IMO, I suggest to create a new function, PyStructSequence_FromModuleAndDesc(module, desc, flags) to create a heaptype and don't aloocates memory block for tp_members,something like 'PyType_FromModuleAndSpec()`.
msg401131 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2021-09-06 11:18
I think you will run into issues with allocating tp_members, because there isn't a good mechanism to for type objects to manage C-level data.

But I encourage you to try, so you get a better understanding of the problem :)
msg401132 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2021-09-06 11:29
> But I encourage you to try, so you get a better understanding of the problem :)

OK, thanks, Petr. I try to add this C API to see what will happen :)
msg402158 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-09-19 18:20
Is the subinterpreters work still on hold pending a PEP?

I thought that conversions to heap types had been suspended because there is a Steering Council level cost benefit decision.  Mostly it seems that everything helps subinterpreters makes code worse in almost every other way (slower, more complex, new APIs, etc).
msg402293 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2021-09-21 08:43
Subinterpreters are not the only reason to do this (and they're not *my* reason to do it).

Adding a way to create PyStructSequence heap will help users of the stable ABI, where reduced performance is a known tradeoff (see https://www.python.org/dev/peps/pep-0652/#stable-abi: "The Stable ABI trades performance for its stability").
More generally, this would need solving one of the remaining limitations of the limited API (PEPs 489, 630): type-specific data. If Hai Shi solves the problem, the solution will be useful even if PyStructSequence_FromModuleAndDesc turns out useless. 

Using the proposed new API in CPython's stdlib should be done much more deliberately, and yes, would need a PEP.
msg402606 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2021-09-25 07:23
> More generally, this would need solving one of the remaining limitations of the limited API (PEPs 489, 630): type-specific data.
Agree. We can't track and destroy the memory block created from the heap now.
History
Date User Action Args
2021-09-26 17:35:26shihai1991setkeywords: + patch
stage: patch review
pull_requests: + pull_request26955
2021-09-25 07:23:46shihai1991setmessages: + msg402606
2021-09-21 08:43:17petr.viktorinsetmessages: + msg402293
2021-09-19 18:20:41rhettingersetnosy: + rhettinger
messages: + msg402158
2021-09-13 09:28:57erlendaaslandsetnosy: + erlendaasland
2021-09-06 16:39:29vstinnersetnosy: - vstinner
2021-09-06 11:29:31shihai1991setmessages: + msg401132
2021-09-06 11:18:09petr.viktorinsetmessages: + msg401131
2021-09-06 11:10:48shihai1991create