classification
Title: [C API] Make PyGC_Head structure opaque, or even move it to the internal C API
Type: Stage: resolved
Components: C API Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: miss-islington, pablogsal, vstinner
Priority: normal Keywords: patch

Created on 2020-04-09 15:19 by vstinner, last changed 2020-06-02 10:09 by miss-islington. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19452 closed vstinner, 2020-04-10 00:16
PR 19461 merged pablogsal, 2020-04-10 18:46
PR 19494 merged vstinner, 2020-04-13 01:00
PR 19499 merged vstinner, 2020-04-13 10:27
PR 19505 merged vstinner, 2020-04-13 23:59
PR 20586 merged vstinner, 2020-06-02 01:27
PR 20593 merged miss-islington, 2020-06-02 10:03
Messages (12)
msg366064 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-09 15:19
Similarly to bpo-39573 (PyObject) and bpo-40170 (PyTypeObject), I propose to make the PyGC_Head structure opaque in the C API.

See https://bugs.python.org/issue39573#msg361513 for the rationale. In short, my plan is to hide all implementation details from the C API.

The PyGC_Head structure caused ABI issues recently: bpo-39599 "ABI breakage between Python 3.7.4 and 3.7.5: change in PyGC_Head structure". Making the structure opaque would reduce the risk of such ABI issue. In fact, the reporter of bpo-39599 really require to access PyGC_Head structure to write a profiler, so this issue doesn't fix all use cases, but it should benefit to most people ;-) PyGC_Head structure will remain accessible via the internal C API which doesn't provide any backward compatibility warranty.
msg366067 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-09 15:20
See also bpo-40240: "Expose public spelling of _PyGC_FINALIZED and _PyGC_SET_FINALIZED?".
msg366092 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-10 00:06
The following macros rely on PyGC_Head:

* _PyGCHead_FINALIZED()
* _PyGCHead_NEXT()
* _PyGCHead_PREV()
* _PyGCHead_SET_FINALIZED()
* _PyGCHead_SET_NEXT()
* _PyGCHead_SET_PREV()
* _PyGC_FINALIZED()
* _PyGC_PREV_MASK
* _PyGC_PREV_MASK_COLLECTING
* _PyGC_PREV_MASK_FINALIZED
* _PyGC_PREV_SHIFT
* _PyGC_SET_FINALIZED()
* _PyObject_GC_IS_TRACKED()
* _PyObject_GC_MAY_BE_TRACKED()
* _Py_AS_GC()

_testcapi uses _PyObject_GC_IS_TRACKED() and sizeof(PyGC_Head).
msg366094 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-10 00:18
> The following macros rely on PyGC_Head: (...)

I already moved them to the internal C API in commit  1a6be91e6fd65ce9cb88cbbbb193db7e92ec6076 of bpo-35081.

But Stefan Behnel reported that it breaks Cython:

"Making _PyGC_FINALIZED() internal broke Cython (https://github.com/cython/cython/issues/2721). It's used in the finaliser implementation (https://github.com/cython/cython/blob/da657c8e326a419cde8ae6ea91be9661b9622504/Cython/Compiler/ModuleNode.py#L1442-L1456), to determine if an object for which tp_dealloc() is called has already been finalised or whether we have to do it. I'm not sure how to deal with this on our side now. Any clue?"

https://bugs.python.org/issue35081#msg330045

So I simply reverted (partially) this change: commit 3e21ad1a254cc33e8d4920ad7f026254ec728bee.
msg366095 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-10 00:23
_testcapi uses _PyObject_GC_IS_TRACKED(). This macro is exposed in Python as gc.is_tracked(). IMO the function should be available in the public C API. For example, PyObject_GC_IsTracked(obj).

Cython uses _PyGC_FINALIZED(). This macro is exposed in Python as gc.is_finalized(). So again, I consider that it should be exposed in a public C function as well, like PyObject_GC_IsFinalized(obj).
msg366170 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-04-11 00:22
New changeset f13072b8a89a922285737988b086beb4b06c6648 by Pablo Galindo in branch 'master':
bpo-40241: Add PyObject_GC_IsTracked and PyObject_GC_IsFinalized to the public C-API (GH-19461)
https://github.com/python/cpython/commit/f13072b8a89a922285737988b086beb4b06c6648
msg366291 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-13 09:38
New changeset 0135598d729d01f35ce08d47160adaa095a6149f by Victor Stinner in branch 'master':
bpo-40241: Add pycore_gc.h header file (GH-19494)
https://github.com/python/cpython/commit/0135598d729d01f35ce08d47160adaa095a6149f
msg366296 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-13 10:20
Ok, this issue should now be closed. It was easier than expected ;-)
msg366300 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-13 10:47
New changeset 0c13e1f96a9487e0efe63c3d3a05ff9738bd7dac by Victor Stinner in branch 'master':
bpo-40241: Add pycore_interp.h header (GH-19499)
https://github.com/python/cpython/commit/0c13e1f96a9487e0efe63c3d3a05ff9738bd7dac
msg366353 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-13 23:59
> bpo-40241: Add pycore_interp.h header (GH-19499)

Oops, this change was for bpo-40268.
msg370600 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-02 10:03
New changeset 337d3103a2344e1fec75985e85fabcbdedac7d26 by Victor Stinner in branch 'master':
bpo-40241: What's New in Python 3.9: opaque PyGC_Head (GH-20586)
https://github.com/python/cpython/commit/337d3103a2344e1fec75985e85fabcbdedac7d26
msg370601 - (view) Author: miss-islington (miss-islington) Date: 2020-06-02 10:09
New changeset ff442f36b91810c8b8a6346e39f1dbf2c6d85764 by Miss Islington (bot) in branch '3.9':
bpo-40241: What's New in Python 3.9: opaque PyGC_Head (GH-20586)
https://github.com/python/cpython/commit/ff442f36b91810c8b8a6346e39f1dbf2c6d85764
History
Date User Action Args
2020-06-02 10:09:35miss-islingtonsetmessages: + msg370601
2020-06-02 10:03:14miss-islingtonsetnosy: + miss-islington

pull_requests: + pull_request19823
2020-06-02 10:03:01vstinnersetmessages: + msg370600
2020-06-02 01:27:49vstinnersetpull_requests: + pull_request19820
2020-04-13 23:59:52vstinnersetpull_requests: + pull_request18856
2020-04-13 23:59:33vstinnersetpull_requests: - pull_request18854
2020-04-13 23:59:27vstinnersetmessages: + msg366353
2020-04-13 23:57:27vstinnersetpull_requests: + pull_request18854
2020-04-13 10:47:21vstinnersetmessages: + msg366300
2020-04-13 10:27:30vstinnersetpull_requests: + pull_request18850
2020-04-13 10:20:38vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg366296

stage: patch review -> resolved
2020-04-13 09:38:46vstinnersetmessages: + msg366291
2020-04-13 01:00:41vstinnersetpull_requests: + pull_request18847
2020-04-11 00:22:06pablogsalsetmessages: + msg366170
2020-04-10 18:46:35pablogsalsetpull_requests: + pull_request18817
2020-04-10 04:22:12ppperrysettitle: [C API] Make PyGC_Head structure opaque, or even more it to the internal C API -> [C API] Make PyGC_Head structure opaque, or even move it to the internal C API
2020-04-10 00:23:41vstinnersetnosy: + pablogsal
messages: + msg366095
2020-04-10 00:18:59vstinnersetmessages: + msg366094
2020-04-10 00:16:12vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request18806
2020-04-10 00:06:57vstinnersetmessages: + msg366092
2020-04-09 15:20:32vstinnersetmessages: + msg366067
2020-04-09 15:19:56vstinnercreate