classification
Title: PyNode_Compile() crashes in Python 3.8.
Type: crash Stage: resolved
Components: Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gvanrossum Nosy List: grahamd, gvanrossum, lukasz.langa, petr.viktorin, vstinner
Priority: release blocker Keywords: patch

Created on 2019-05-28 01:52 by grahamd, last changed 2019-05-28 23:45 by gvanrossum. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 13634 merged gvanrossum, 2019-05-28 18:17
Messages (6)
msg343727 - (view) Author: Graham Dumpleton (grahamd) Date: 2019-05-28 01:52
The code:

#include <Python.h>

int
main(int argc, char *argv[])
{
    FILE *fp = NULL;
    PyObject *co = NULL;
    struct _node *n = NULL;
    const char * filename = "/dev/null";

    Py_Initialize();

    fprintf(stderr, "START\n");

    fp = fopen(filename, "r");

    fprintf(stderr, "CALL PyParser_SimpleParseFile()\n");

    n = PyParser_SimpleParseFile(fp, filename, Py_file_input);

    fprintf(stderr, "CALL PyNode_Compile()\n");

    co = (PyObject *)PyNode_Compile(n, filename);

    fprintf(stderr, "DONE\n");

    Py_Finalize();

    return 0;
}

has worked fine since Python 2.3 (and maybe earlier) through Python 3.7, but now crashes in Python 3.8.

It crashes in PyNode_Compile().

START
CALL PyParser_SimpleParseFile()
CALL PyNode_Compile()
Segmentation fault: 11

Although it is part of the public interface of compile.h, the PyNode_Compile() seems never to actually be called anywhere in Python itself, and perhaps isn't even covered by tests. So if Python 3.8 internal changes mean this function implementation needs to be changed, that fact may have been missed.
msg343728 - (view) Author: Graham Dumpleton (grahamd) Date: 2019-05-28 02:10
FWIW, this was occurring on macOS. Not been able to test on other platforms.
msg343750 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2019-05-28 08:42
Looks like this is caused by: https://github.com/python/cpython/pull/12086/files#diff-4d35cf8992b795c5e97e9c8b6167cb34R787

PyAST_FromNodeObject doesn't ignore flags any more, so when PyNode_Compile passes NULL flags, it crashes.

(This is unfamiliar code for me; I won't have time to fix & test properly this week.)
msg343761 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-05-28 10:33
I mark this issue as a release blocker.
msg343811 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2019-05-28 18:03
OK, I'll look into this.
msg343834 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2019-05-28 23:45
New changeset 77f0ed7a42606d03ebfe48ab152caf0d796d6540 by Guido van Rossum in branch 'master':
bpo-37072: Fix crash in PyAST_FromNodeObject() when flags is NULL (#13634)
https://github.com/python/cpython/commit/77f0ed7a42606d03ebfe48ab152caf0d796d6540
History
Date User Action Args
2019-05-28 23:45:50gvanrossumsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-05-28 23:45:01gvanrossumsetmessages: + msg343834
2019-05-28 18:17:10gvanrossumsetkeywords: + patch
stage: patch review
pull_requests: + pull_request13534
2019-05-28 18:03:33gvanrossumsetassignee: gvanrossum

messages: + msg343811
nosy: + gvanrossum
2019-05-28 10:33:11vstinnersetpriority: normal -> release blocker
nosy: + vstinner, lukasz.langa
messages: + msg343761

2019-05-28 08:42:07petr.viktorinsetnosy: + petr.viktorin
messages: + msg343750
2019-05-28 02:10:44grahamdsettype: crash
messages: + msg343728
2019-05-28 01:52:54grahamdcreate