Title: PyArg_ParseTuple() false-returns SUCCESS though SystemError and missing data (when PY_SSIZE_T_CLEAN not #define'd)
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.10
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: kxrob, methane, vstinner
Priority: normal Keywords: patch

Created on 2021-02-25 16:05 by kxrob, last changed 2021-03-08 22:02 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 24656 merged methane, 2021-02-26 04:22
Messages (5)
msg387678 - (view) Author: Robert (kxrob) * Date: 2021-02-25 16:05
When PY_SSIZE_T_CLEAN  is not #defined in Py3.10, PyArg_ParseTuple() etc. sets a SystemError but the return value says 1 (=SUCCESS)! 
=>  Causes terrific crashes with unfilled variables - instead of a clean Python exception.

Background: pywin32 suffers in masses from the drop of int support for the s#/y#/et#... formats in PyArg_ParseTuple() etc.  (PY_SSIZE_T_CLEAN is required in Py3.10). And only part of the source is already PY_SSIZE_T_CLEAN. Now it is very difficult to even run tests and weed out / check, because of freezes instead of nice Python exceptions.

=> Instead of preventing such freezes, the PY_SSIZE_T_CLEAN mechanism became the opposite: a clever trap, a sword of Damocles :)

The cause is in getargs.c:

=================== getargs.c / convertsimple() ====

    if (!(flags & FLAG_SIZE_T)) { \
        PyErr_SetString(PyExc_SystemError, \
                        "PY_SSIZE_T_CLEAN macro must be defined for '#' formats"); \
        return NULL; \
#define RETURN_ERR_OCCURRED return msgbuf


=> The return NULL is further processed as no msg NULL -> no error.  
=> Perhaps it should be a `return converterr(...)` or `return sstc_system_error(...)` !?
msg387696 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2021-02-26 00:46
Thank you for reporting.

@vstinner Can we provide a nice C traceback too?
msg387706 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2021-02-26 04:16
I checked the warning, and now I think Python traceback is fine.

PyArg_Parse*() is used on top of the C function. So python traceback is enough to find which function is using '#' without Py_SSIZE_T_CLEAN.
msg387776 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2021-02-27 11:31
New changeset c71d24f55828e7f0f2c8750d2e1b04d04539beff by Inada Naoki in branch 'master':
bpo-43321: Fix SystemError in getargs.c (GH-24656)
msg388312 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-03-08 22:02
Thanks for fixing my bug ;-) (bpo-40943)
Date User Action Args
2021-03-08 22:02:34vstinnersetmessages: + msg388312
2021-02-27 11:31:31methanesetmessages: + msg387776
2021-02-27 11:31:19methanesetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-02-26 04:22:11methanesetkeywords: + patch
stage: patch review
pull_requests: + pull_request23441
2021-02-26 04:16:10methanesetmessages: + msg387706
2021-02-26 00:46:31methanesetnosy: + vstinner
messages: + msg387696
2021-02-26 00:36:19methanesetnosy: + methane
2021-02-25 16:05:25kxrobcreate