New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Possible integer overflow in PyCode_New() #62495
Comments
On Windows x64, we get the following warning: ..\Objects\codeobject.c(106): warning C4244: '=' : conversion from 'Py_ssize_t' to 'unsigned char', possible loss of data [C:\buildbot.python.org\3.x.kloth-win64\build\PCbuild\pythoncore.vcxproj] Code: unsigned char *cell2arg = NULL;
Py_ssize_t total_args = argcount + kwonlyargcount +
((flags & CO_VARARGS) != 0) + ((flags & CO_VARKEYWORDS) != 0);
PyObject *cell = PyTuple_GET_ITEM(cellvars, i);
for (j = 0; j < total_args; j++) {
PyObject *arg = PyTuple_GET_ITEM(varnames, j);
if (!PyUnicode_Compare(cell, arg)) {
====> cell2arg[i] = j; <===== HERE
used_cell2arg = 1;
break;
}
} total_args is not checked for being smaller than 256. Related issue: bpo-9566. |
Similar issue: ..\Objects\funcobject.c(636): warning C4244: 'function' : conversion from 'Py_ssize_t' to 'int', possible loss of data [C:\buildbot.python.org\3.x.kloth-win64\build\PCbuild\pythoncore.vcxproj] Extract of function_call() function: result = PyEval_EvalCodeEx(
PyFunction_GET_CODE(func),
PyFunction_GET_GLOBALS(func), (PyObject *)NULL,
&PyTuple_GET_ITEM(arg, 0), PyTuple_GET_SIZE(arg),
k, nk, d, nd,
PyFunction_GET_KW_DEFAULTS(func),
PyFunction_GET_CLOSURE(func)); argcount, kwcount and defcount are int, whereas function_call() pass Py_ssize_t values. function_call() should check PyTuple_GET_SIZE(arg) <= INT_MAX, nk <= INT_MAX and nd <= INT_MAX. |
And another one: ..\Python\ceval.c(4271): warning C4244: '=' : conversion from 'Py_ssize_t' to 'int', possible loss of data [C:\buildbot.python.org\3.x.kloth-win64\build\PCbuild\pythoncore.vcxproj] First in fast_function(), nd type is int: if (argdefs != NULL) {
d = &PyTuple_GET_ITEM(argdefs, 0);
==> nd = Py_SIZE(argdefs); <=== HERE
}
return PyEval_EvalCodeEx((PyObject*)co, globals,
(PyObject *)NULL, (*pp_stack)-n, na,
(*pp_stack)-2*nk, nk, d, nd, kwdefs,
PyFunction_GET_CLOSURE(func)); Second in ext_do_call(), nstar type is int: nstar = PyTuple_GET_SIZE(stararg); Must check: Py_SIZE(argdefs) <= INT_MAX and PyTuple_GET_SIZE(stararg) <= INT_MAX. |
I don't think they are actually the *same* issue. For the limitations wrt. code objects (maximum size of byte code, maximum number of local variables, maximum number of parameters, etc.), I recommend the following thorough procedure:
|
Here is a patch adding Py_SAFE_DOWNCAST(). For update_star_args(), I changed the type instead, because I prefer to avoid Py_SAFE_DOWNCAST() when possible. Modify PyEval_EvalCodeEx() and PyCode_New() to use Py_ssize_t would be more correct, but it may be slower if Py_ssize_t is larger than int, and I hope that nobody calls functions with more than INT_MAX parameters! It would be completly inefficient! |
Note this is referenced from bpo-18407. |
Many of these overflows can be provoked by specially constructed function, code object or bytecode. Also I think following examples crash or return wrong result on 64 bit platform: def f(*args, **kwargs): return len(args), len(kwargs)
f(*([0]*(2**32+1)))
f(**dict.fromkeys(map(hex, range(2**31+1)))) Here is updated patch which handles overflows in non-debug build. It prevent creating Python function with more than 255 default values (in any case compiler and interpreter don't support more than 255 arguments) and raise exception when function is called with too many arguments or too large *args or **kwargs. |
Other than my one query on the review, code_ssize_t_2.patch.patch looks good to me. |
It is possible to reproduce original bug without hacking the code object or bytecode: >>> eval('lambda %s, *args, **kwargs: (lambda:args)' % (', '.join('a%d'%i for i in range(253)),))(*range(256))()
(253, 254, 255)
>>> eval('lambda %s, *args, **kwargs: (lambda:args)' % (', '.join('a%d'%i for i in range(254)),))(*range(256))()
(254, 255)
>>> eval('lambda %s, *args, **kwargs: (lambda:args)' % (', '.join('a%d'%i for i in range(255)),))(*range(256))()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1, in <lambda>
NameError: free variable 'args' referenced before assignment in enclosing scope
>>> eval('lambda %s, *args, **kwargs: (lambda:kwargs)' % (', '.join('a%d'%i for i in range(253)),))(*range(256))()
{}
>>> eval('lambda %s, *args, **kwargs: (lambda:kwargs)' % (', '.join('a%d'%i for i in range(254)),))(*range(256))()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1, in <lambda>
NameError: free variable 'kwargs' referenced before assignment in enclosing scope
>>> eval('lambda %s, *args, **kwargs: (lambda:kwargs)' % (', '.join('a%d'%i for i in range(255)),))(*range(256))()
0 |
Could we try and get this closed please, as I'm always a little concerned that a code change causes a genuine warning that should be actioned, but it gets masked by all the others. |
New changeset e615718a6455 by Victor Stinner in branch 'default': |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: