New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PyEval_EvalCodeEx() can no longer be called with code which has (CO_NEWLOCALS | CO_OPTIMIZED) flags #87707
Comments
Cython generates a __Pyx_PyFunction_FastCallDict() function which calls PyEval_EvalCodeEx(). With Python 3.9, it worked well. With Python 3.10 in debug mode, it fails with an assertion error: python3.10: Python/ceval.c:5148: PyEval_EvalCodeEx: Assertion `(((PyCodeObject *)_co)->co_flags & (CO_NEWLOCALS | CO_OPTIMIZED)) == 0' failed. With Python 3.10 in release mode, it does crash. Context of the failed assertion:
See the Cython issue to a reproducer: cython/cython#4025 (comment) In Python 3.9, _PyFunction_Vectorcall() has the following fast-path: if (co->co_kwonlyargcount == 0 && nkwargs == 0 &&
(co->co_flags & ~PyCF_MASK) == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE))
{
if (argdefs == NULL && co->co_argcount == nargs) {
return function_code_fastcall(tstate, co, stack, nargs, globals);
}
else if (nargs == 0 && argdefs != NULL
&& co->co_argcount == PyTuple_GET_SIZE(argdefs)) {
/* function called with no arguments, but all parameters have
a default value: use default values as arguments .*/
stack = _PyTuple_ITEMS(argdefs);
return function_code_fastcall(tstate, co,
stack, PyTuple_GET_SIZE(argdefs),
globals);
}
} When the bug occurs, __Pyx_PyFunction_FastCallDict() doesn't take the fast-path because nargs < co_argcount (1 < 2). In Python 3.10, _PyFunction_Vectorcall() is very different: if (((PyCodeObject *)f->fc_code)->co_flags & CO_OPTIMIZED) {
return _PyEval_Vector(tstate, f, NULL, stack, nargs, kwnames);
}
else {
return _PyEval_Vector(tstate, f, f->fc_globals, stack, nargs, kwnames);
} PyEval_EvalCodeEx() must not crash if the code object has (CO_NEWLOCALS | CO_OPTIMIZED | CO_NOFREE) flags. |
If I the comment the following assertion in PyEval_EvalCodeEx(): assert ((((PyCodeObject *)_co)->co_flags & (CO_NEWLOCALS | CO_OPTIMIZED)) == 0); The garbage collector fails while visiting builtins of the function with func_traverse(), at line: Py_VISIT(f->func_builtins); I guess that the problem is that f->func_builtins doesn't hold a strong reference to builtins. Error: Modules/gcmodule.c:113: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small object address : 0x7fffea69d650 Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed Current thread 0x00007ffff7c20740 (most recent call first): Extension modules: Cython.Plex.Actions, Cython.Plex.Scanners, Cython.Compiler.Scanning, Cython.Tempita._tempita, Cython.Compiler.Visitor, Cython.Compiler.FlowControl (total: 6) Program received signal SIGABRT, Aborted. |
In commit 46496f9, I modified _PyEval_BuiltinsFromGlobals() to return a borrowed reference rather than a strong reference. It seems like I forgot to remove a Py_DECREF() in PyEval_EvalCodeEx()! I'm working on a fix. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: