classification
Title: Remove Py_TPFLAGS_HAVE_VERSION_TAG flag?
Type: Stage: patch review
Components: C API Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: corona10, miss-islington, petr.viktorin, pitrou, serhiy.storchaka, shihai1991, vstinner
Priority: normal Keywords: patch

Created on 2020-12-25 23:48 by vstinner, last changed 2021-07-23 13:21 by miss-islington.

Pull Requests
URL Status Linked Edit
PR 27260 merged petr.viktorin, 2021-07-20 14:40
PR 27306 open miss-islington, 2021-07-23 13:21
Messages (11)
msg383782 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-12-25 23:48
Since the PyTypeObject structure is excluded from the limited C API and the stable ABI on purpose (PEP 384), I don't see the purpose of the Py_TPFLAGS_HAVE_VERSION_TAG flag.

Moreover, a new flag was added recently:

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030A0000
/* Type has am_send entry in tp_as_async slot */
#define Py_TPFLAGS_HAVE_AM_SEND (1UL << 21)
#endif

Should it be also removed?


For example, Py_TPFLAGS_HAVE_FINALIZE was deprecated in bpo-32388 by:

commit ada319bb6d0ebcc68d3e0ef2b4279ea061877ac8
Author: Antoine Pitrou <antoine@python.org>
Date:   Wed May 29 22:12:38 2019 +0200

    bpo-32388: Remove cross-version binary compatibility requirement in tp_flags (GH-4944)
    
    It is now allowed to add new fields at the end of the PyTypeObject struct without having to allocate a dedicated compatibility flag i
n tp_flags.
    
    This will reduce the risk of running out of bits in the 32-bit tp_flags value.


By the way, is it worth it to remove Py_TPFLAGS_HAVE_FINALIZE? Or is it going to break too many extension modules?
msg383809 - (view) Author: Dong-hee Na (corona10) * (Python committer) Date: 2020-12-26 16:00
Let's emit the deprecated message if Py_TPFLAGS_HAVE_VERSION_TAG is set ;)
For example, PyType_Ready looks like a good place
msg383821 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-12-26 19:07
Maybe we can keep the constants but define them as 0 and deprecate them.

The question is more if these constants solve a real ABI issue or not.
msg383855 - (view) Author: hai shi (shihai1991) * (Python triager) Date: 2020-12-27 17:45
>I don't see the purpose of the Py_TPFLAGS_HAVE_VERSION_TAG flag.

IMO, There have no more exact description of `Py_TPFLAGS_HAVE_VERSION_TAG`in docs, so I perfer to remove it.

> By the way, is it worth it to remove Py_TPFLAGS_HAVE_FINALIZE? Or is it going to break too many extension modules?

I am not found who use Py_TPFLAGS_HAVE_AM_SEND or Py_TPFLAGS_HAVE_FINALIZE in github.I perfer to keep it as soon as posssible if there have users using them.
MAYBE we should do some surveys firstly?
msg383971 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2020-12-29 10:12
IMO, these flags are useless. Both the stable ABI and the version-specific builds of extension modules use the memory layout of the current interpreter and fill unset slots with NULL.

Py_TPFLAGS_HAVE_AM_SEND is new in 3.10, so it can  be removed (or replaced by a check for Py_TYPE(iter)->tp_as_async != NULL && Py_TYPE(iter)->tp_as_async->am_send != NULL).

Py_TPFLAGS_HAVE_VERSION_TAG and Py_TPFLAGS_HAVE_FINALIZE are there since 2.6 and 3.8 respectively, and mentioned in documentation. Extensions that use the stable ABI can *check for them*. We cannot do a survey of all existing extension modules. The flags can't be removed without breaking stable ABI.
msg383974 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-12-29 10:36
I don't think that it was right thing to break binary compatibility. It virtually buried the stable ABI.
msg384040 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-12-30 01:10
Serhiy Storchaka:
> I don't think that it was right thing to break binary compatibility. It virtually buried the stable ABI.

IMO there is a misunderstanding about the stable ABI. PyType_FromSpec() doesn't need Py_TPFLAGS_HAVE_xxx flags: this function allocates a heap type which *has* all these members (set to 0/NULL by default).

And it's not possible to define a static type using the limited C API, since the PyTypeObject structure is excluded from it on purpose.

See bpo-32388 for a similar discussion.
msg384041 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-12-30 01:22
Ah, I see that there is a misunderstanding. The flag is used for two things:

(*) Create/Declare a type

IMO Py_TPFLAGS_HAVE_VERSION_TAG and Py_TPFLAGS_HAVE_FINALIZE are useless for that.

(*) Check if a type has the flag

An extension built with the stable ABI *can* use PyType_HasFeature(type, Py_TPFLAGS_HAVE_VERSION_TAG).

I'm only aware of PyQt which uses the stable ABI. Is there other projects using it? Do these projects check for Py_TPFLAGS_HAVE_VERSION_TAG or Py_TPFLAGS_HAVE_FINALIZE flag?

If we cannot be used that no extension module built with the stable ABI for these flags, the safe option is to keep them, deprecate them, and always define them. For example, add the flags to Py_TPFLAGS_DEFAULT and/or add them in type_new() or PyType_Ready().

--

Py_TPFLAGS_HAVE_VERSION_TAG is really an internal tag. It should only be used by _PyType_Lookup().

I'm not sure why someone would like to check explicitly for the Py_TPFLAGS_HAVE_FINALIZE flag. I would expect that only Python internals call the tp_finalize slot.
msg384051 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2020-12-30 08:50
> I'm only aware of PyQt which uses the stable ABI. Is there other projects using it? Do these projects check for Py_TPFLAGS_HAVE_VERSION_TAG or Py_TPFLAGS_HAVE_FINALIZE flag?

I think there are. Who knows how many; they're not required to register anywhere.


If the flags do end up being removed, it would be nice to reserve the bits (in a comment), so they aren't reused too soon.

Also, ISTM that PEP 387 applies, and requires warning for two releases. How can that be done for flags?
msg397873 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2021-07-20 14:25
The bit cannot be repurposed, since older extensions using the stable ABI might set it.
It doesn't make much sense to remove the Py_TPFLAGS_HAVE_VERSION_TAG or Py_TPFLAGS_HAVE_FINALIZE defines; I'd let them stay to document that the bits are reserved.
msg398050 - (view) Author: miss-islington (miss-islington) Date: 2021-07-23 13:21
New changeset a4760cc32d9e5dac7be262e9736eb30502cd7be3 by Petr Viktorin in branch 'main':
bpo-42747: Remove Py_TPFLAGS_HAVE_AM_SEND and make Py_TPFLAGS_HAVE_VERSION_TAG no-op (GH-27260)
https://github.com/python/cpython/commit/a4760cc32d9e5dac7be262e9736eb30502cd7be3
History
Date User Action Args
2021-07-23 13:21:24miss-islingtonsetpull_requests: + pull_request25850
2021-07-23 13:21:20miss-islingtonsetnosy: + miss-islington
messages: + msg398050
2021-07-20 14:40:48petr.viktorinsetkeywords: + patch
stage: patch review
pull_requests: + pull_request25805
2021-07-20 14:25:29petr.viktorinsetmessages: + msg397873
2020-12-30 08:50:07petr.viktorinsetmessages: + msg384051
2020-12-30 01:22:42vstinnersetmessages: + msg384041
2020-12-30 01:10:10vstinnersetmessages: + msg384040
2020-12-29 10:36:22serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg383974
2020-12-29 10:12:04petr.viktorinsetmessages: + msg383971
2020-12-27 17:45:58shihai1991setnosy: + shihai1991
messages: + msg383855
2020-12-26 19:07:34vstinnersetmessages: + msg383821
2020-12-26 16:00:09corona10setnosy: + corona10
messages: + msg383809
2020-12-25 23:48:07vstinnercreate