classification
Title: PyStructSequence_NewType broken in 3.8
Type: crash Stage: resolved
Components: C API Versions: Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: kj, lukasz.langa, miss-islington, petr.viktorin, stestagg, wdi2
Priority: high Keywords: patch

Created on 2020-10-19 18:27 by wdi2, last changed 2021-05-04 14:23 by kj. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 25852 closed kj, 2021-05-03 14:06
PR 25886 merged kj, 2021-05-04 10:13
PR 25887 merged miss-islington, 2021-05-04 11:08
PR 25896 merged petr.viktorin, 2021-05-04 13:25
Messages (6)
msg378978 - (view) Author: Xemistry GmbH (wdi2) Date: 2020-10-19 18:27
Calling PyStructSequence_NewType() with a NULL field in the desc.doc parameter (explicitly allowed as per docs) leads to a crash in 

Objects/typeobject.c:2956
2956                size_t len = strlen(old_doc)+1;

where old_doc is NULL.
If the doc string is set, the call succeeds, but with a warning

(stdin):1: DeprecationWarning: builtin type G_SGROUP has no __module__ attribute

(where G_SGROUP is my new type), which did not happen in 3.6, and which I do not think can be suppressed by function call arguments.
msg382853 - (view) Author: Steve Stagg (stestagg) Date: 2020-12-10 23:12
It looks like the segfault was fixed in https://github.com/python/cpython/commit/88c2cfd9ffbcfc43fd1364f2984852a819547d43

as part of https://bugs.python.org/issue41832.

The code in this area of typeobject.c looks a bit different, now, but the backport seems simple?

Simple testcase:

#include <stdio.h>
#include <Python.h>
int main() {
    Py_Initialize();
    PyStructSequence_Field fields[2] = {
        {NULL, NULL}
    };
    PyStructSequence_Desc d = {"test", NULL, &fields[0], 0};
    PyStructSequence_NewType(&d);
    Py_Finalize();
}

Segfault reproducible on 3.8 and 3.9
msg392686 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-05-02 10:52
This missed the train for inclusion in 3.8. There's still time for a backport for 3.9.
msg392893 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2021-05-04 13:25
Changing PyType_FromSpec* to accept NULL has an issue: extensions built and tested with 3.9.5 would not work with the earlier 3.9s.

I'll send a PR to fix just PyStructSequence_NewType.
msg392899 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-05-04 14:07
New changeset ec18362f6a7fdc02f9f982872fc1006bca31627d by Petr Viktorin in branch '3.9':
[3.9] bpo-42083: Allow NULL doc in PyStructSequence_NewType (#25896)
https://github.com/python/cpython/commit/ec18362f6a7fdc02f9f982872fc1006bca31627d
msg392901 - (view) Author: Ken Jin (kj) * (Python triager) Date: 2021-05-04 14:23
Steve, thank you for your invaluable investigation. Thanks Petr for a better fix - your issue didn't come to my mind at the time.

Since all PRs have landed and the fix should arrive in Python 3.9.6, I am closing this issue. Please don't hesitate to reopen this if anyone feels it needs revisiting. Thanks!
History
Date User Action Args
2021-05-04 14:23:08kjsetstatus: open -> closed
resolution: fixed
messages: + msg392901

stage: patch review -> resolved
2021-05-04 14:07:20lukasz.langasetmessages: + msg392899
2021-05-04 13:25:53petr.viktorinsetmessages: + msg392893
2021-05-04 13:25:06petr.viktorinsetnosy: + petr.viktorin
pull_requests: + pull_request24568
2021-05-04 11:08:48miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request24561
2021-05-04 10:13:16kjsetpull_requests: + pull_request24560
2021-05-03 14:06:51kjsetkeywords: + patch
nosy: + kj

pull_requests: + pull_request24536
stage: needs patch -> patch review
2021-05-02 10:52:59lukasz.langasetversions: + Python 3.9, Python 3.10, - Python 3.8
nosy: + lukasz.langa

messages: + msg392686

stage: needs patch
2020-12-10 23:12:47stestaggsetnosy: + stestagg
messages: + msg382853
2020-10-19 19:14:31rhettingersetpriority: normal -> high
2020-10-19 18:27:05wdi2create