The following code results in a segmentation fault in CPython 3.8 while executes fines (raises a SystemError) in CPython 3.7.

PyObject* debug(PyObject *self, PyObject *args)
{
        const char * debug = "debug";
        PyErr_SetString(PyExc_ValueError, "debug!");
        return Py_BuildValue("(s#O)", debug, strlen(debug), Py_None);
}

It seems necessary for the format string to contain both an instance of "s#" and "O" to trigger the bug.

I'm attaching a C module that reproduces the problem.

seen with the current 3.8 branch.

Does it depend on whether PY_SSIZE_T_CLEAN is defined?

It is not reproducible with PY_SSIZE_T_CLEAN defined.

The problem arises from this code in do_mktuple(), staring at line 394 in modsupport.c:

            if (**p_format == '#') {
                ++*p_format;
                if (flags & FLAG_SIZE_T)
                    n = va_arg(*p_va, Py_ssize_t);
                else {
                    if (PyErr_WarnEx(PyExc_DeprecationWarning,
                                "PY_SSIZE_T_CLEAN will be required for '#' formats", 1)) {
                        return NULL;
                    }
                    n = va_arg(*p_va, int);
                }
            }

If this is entered with an exception raised, PyErr_WarnEx() return NULL, thus this function return NULL without consuming the argument relative to the string length for the "s#" specifier. This argument is then consumed at the next iteration for the "O" specifier, resulting in a segmentation fault when the string length is interpreted as an object pointer.

I don't know what is the best solution: either ignoring the return value of PyErr_WarnEx or swapping the lines from

                    if (PyErr_WarnEx(PyExc_DeprecationWarning,
                                "PY_SSIZE_T_CLEAN will be required for '#' formats", 1)) {
                        return NULL;
                    }
                    n = va_arg(*p_va, int);

to

                    n = va_arg(*p_va, int);
                    if (PyErr_WarnEx(PyExc_DeprecationWarning,
                                "PY_SSIZE_T_CLEAN will be required for '#' formats", 1)) {
                        return NULL;
                    }

The handling of the "y#" just below suffers from the same problem.

Note: this is going to miss Python 3.8.1 as I'm releasing 3.8.1rc1 right now. Please address this before 3.8.2 (due in February).

Sadly, this missed the train for 3.8.2 as well.

New changeset 28d0bca by Serhiy Storchaka in branch 'master':
bpo-38913: Fix segfault in Py_BuildValue("(s#O)", ...) if entered with exception raised. (GH-18656)
28d0bca

New changeset a7b8a96 by Serhiy Storchaka in branch '3.8':
[3.8] bpo-38913: Fix segfault in Py_BuildValue("(sGH-O)", ...) if entered with exception raised. (GH-18656). (GH-18732)
a7b8a96

After this change, the arm64 buildbots are reporting reference leaks:

1:03:24 load avg: 0.95 Re-running failed tests in verbose mode
1:03:24 load avg: 0.95 Re-running test_capi in verbose mode
test_capi leaked [4, 4, 4] references, sum=12

e.g. https://buildbot.python.org/all/#/builders/563/builds/25

Actually there is a leak in PyErr_WarnEx().

Opened bpo-39831.