This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: types.GenericAlias should decref instead of using delete in tp_new
Type: Stage: resolved
Components: Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: kj, miss-islington, pablogsal, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2021-07-04 15:34 by kj, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27016 merged kj, 2021-07-04 15:42
PR 27018 merged miss-islington, 2021-07-04 16:53
PR 27019 merged miss-islington, 2021-07-04 16:54
PR 27021 merged pablogsal, 2021-07-04 19:01
PR 27022 merged pablogsal, 2021-07-04 19:08
PR 27028 merged kj, 2021-07-05 09:03
PR 27031 merged miss-islington, 2021-07-05 11:11
Messages (15)
msg396944 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-07-04 15:34
Ref: comment chain at https://github.com/python/cpython/pull/27008#discussion_r663450441.

setup_ga fails only if PyTuple_Pack fails, which usually happens when Python is out of memory. The cleanup code for setup_ga shouldn't use PyObject_GC_DEL as mentioned in Pablo's comment above.

I don't know how to add a test for out of memory, so I'll send a PR without test soon.
msg396945 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-04 16:47
New changeset d33943a6c368c2184e238019c63ac7a267da5594 by Ken Jin in branch 'main':
bpo-44562: Remove invalid PyObject_GC_Del from error path of types.GenericAlias … (GH-27016)
https://github.com/python/cpython/commit/d33943a6c368c2184e238019c63ac7a267da5594
msg396947 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-07-04 17:53
There is the same bug in ga_new(): tp_free is called directly.

And is is correct to call deallocator for not initialized GenericAlias object?
msg396948 - (view) Author: miss-islington (miss-islington) Date: 2021-07-04 17:55
New changeset 68330b681a4b63cedad58fcfd1d9573209bad21d by Miss Islington (bot) in branch '3.10':
bpo-44562: Remove invalid PyObject_GC_Del from error path of types.GenericAlias … (GH-27016)
https://github.com/python/cpython/commit/68330b681a4b63cedad58fcfd1d9573209bad21d
msg396949 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-04 17:55
New changeset 4684a34c8d2a2ffac7b779edb4ba57f043783478 by Miss Islington (bot) in branch '3.9':
bpo-44562: Remove invalid PyObject_GC_Del from error path of types.GenericAlias … (GH-27016) (GH-27018)
https://github.com/python/cpython/commit/4684a34c8d2a2ffac7b779edb4ba57f043783478
msg396951 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-04 18:05
> And is is correct to call deallocator for not initialized GenericAlias object?

The correct thing to do is to call Py_DECREF so the object is properly unlinked and the other potential cleanups work. Calling the deallocator directly is almost always dangerous.
msg396953 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-07-04 18:19
But is alias->origin initialized to safely call Py_XDECREF(alias->origin)?
msg396954 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-04 18:28
> But is alias->origin initialized to safely call Py_XDECREF(alias->origin)?

Why would that be unsafe? PyType_GenericAlloc sets all fields to NULL so that is going to either be NULL or have an object, no? setup_ga only sets alias->origin if there has been no failures
msg396955 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-07-04 18:47
AFAIK PyType_GenericAlloc is used indirectly in ga_new(), but not in Py_GenericAlias().
msg396956 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-04 18:53
> AFAIK PyType_GenericAlloc is used indirectly in ga_new(), but not in Py_GenericAlias().

Oh, I see what you mean, that's a good point.
msg396959 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-04 19:26
New changeset fe847a62852c3baaec6c97a5e2e7b2e66732bdb8 by Pablo Galindo in branch '3.9':
Revert "bpo-44562: Remove invalid PyObject_GC_Del from error path of types.GenericAlias … (GH-27016) (GH-27018)" (GH-27022)
https://github.com/python/cpython/commit/fe847a62852c3baaec6c97a5e2e7b2e66732bdb8
msg396973 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-07-05 08:52
> Revert "bpo-44562: Remove invalid PyObject_GC_Del from error path of types.GenericAlias … (GH-27016) (GH-27018)" (GH-27022)

That's a good idea :-)

The reverted commit caused many segfaults on macOS:
https://buildbot.python.org/all/#/builders/108/builds/122

4 tests failed:
    test_cmd_line_script test_multiprocessing_main_handling
    test_tarfile test_trace
msg396987 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-05 11:10
New changeset b324c4c5f763c5116a97db8591e6dcb94456570a by Pablo Galindo in branch 'main':
bpo-44562: Use PyType_GenericAlloc in Py_GenericAlias (GH-27021)
https://github.com/python/cpython/commit/b324c4c5f763c5116a97db8591e6dcb94456570a
msg396988 - (view) Author: miss-islington (miss-islington) Date: 2021-07-05 11:34
New changeset d17cc1ff9fe82b17bbe589b83e440959c8f135d7 by Miss Islington (bot) in branch '3.10':
bpo-44562: Use PyType_GenericAlloc in Py_GenericAlias (GH-27021)
https://github.com/python/cpython/commit/d17cc1ff9fe82b17bbe589b83e440959c8f135d7
msg397003 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-05 16:22
New changeset 51a29c42f10bd9368db9a21f2f63319be2e30b95 by Ken Jin in branch '3.9':
[3.9] bpo-44562: Remove invalid PyObject_GC_Del from error path of types.GenericAlias … (GH-27016) (GH-27028)
https://github.com/python/cpython/commit/51a29c42f10bd9368db9a21f2f63319be2e30b95
History
Date User Action Args
2022-04-11 14:59:47adminsetgithub: 88728
2021-07-05 16:34:14pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-07-05 16:22:51pablogsalsetmessages: + msg397003
2021-07-05 11:34:30miss-islingtonsetmessages: + msg396988
2021-07-05 11:11:03miss-islingtonsetpull_requests: + pull_request25591
2021-07-05 11:10:57pablogsalsetmessages: + msg396987
2021-07-05 09:03:33kjsetpull_requests: + pull_request25587
2021-07-05 08:52:50vstinnersetnosy: + vstinner
messages: + msg396973
2021-07-04 19:26:43pablogsalsetmessages: + msg396959
2021-07-04 19:08:27pablogsalsetpull_requests: + pull_request25581
2021-07-04 19:01:48pablogsalsetpull_requests: + pull_request25580
2021-07-04 18:53:36pablogsalsetmessages: + msg396956
2021-07-04 18:47:17serhiy.storchakasetmessages: + msg396955
2021-07-04 18:28:51pablogsalsetmessages: + msg396954
2021-07-04 18:19:00serhiy.storchakasetmessages: + msg396953
2021-07-04 18:05:11pablogsalsetmessages: + msg396951
2021-07-04 17:55:46pablogsalsetmessages: + msg396949
2021-07-04 17:55:42miss-islingtonsetmessages: + msg396948
2021-07-04 17:53:59serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg396947
2021-07-04 16:54:01miss-islingtonsetpull_requests: + pull_request25578
2021-07-04 16:53:56miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25577
2021-07-04 16:47:44pablogsalsetmessages: + msg396945
2021-07-04 15:42:19kjsetkeywords: + patch
stage: patch review
pull_requests: + pull_request25575
2021-07-04 15:34:00kjcreate