New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
int arithmetic relies on C signed overflow behaviour #51655
Comments
Much of the code in Objects/intobject.c assumes that an arithmetic An obvious example is found in int_add, which looks like this: static PyObject *
int_add(PyIntObject *v, PyIntObject *w)
{
register long a, b, x;
CONVERT_TO_LONG(v, a);
CONVERT_TO_LONG(w, b);
x = a + b;
if ((x^a) >= 0 || (x^b) >= 0)
return PyInt_FromLong(x);
return PyLong_Type.tp_as_number->nb_add((PyObject *)v, (PyObject
*)w);
} Here Python is relying on the line 'x = a + b' wrapping on overflow. More generally, Python's source makes a number of assumptions about
(Relevant standard sections: C99 6.2.6.2, C99 6.3.1.3p3.) See also bpo-1621. |
Fixed int_sub, int_add, int_mul, and the fast paths for BINARY_ADD and |
Here is a way to test for overflow which is correct for any C implementation: static PyObject *
int_add(PyIntObject *v, PyIntObject *w)
{
register long a, b;
CONVERT_TO_LONG(v, a);
CONVERT_TO_LONG(w, b);
if (((a>0)&&(b>0)&&((LONG_MAX-a)<b))
||((a<0)&&(b<0)&&((LONG_MIN-a)>b))) {
/* would overflow the long type */
return PyLong_Type.tp_as_number->nb_add((PyObject *)v, (PyObject *)w);
}
return PyInt_FromLong(a+b);
} |
Here is a set of macros that I use to test for overflow: http://allmydata.org/trac/libzutil/browser/libzutil/zutilimp.h |
Zooko: Yes; that's the sort of solution that's needed if we're not Also, if we're not allowed to assume two's complement + no trap For ones' complement or sign-and-magnitude, the result of these For two's complement with (-sys.maxint-1) a trap representation, That's why I want to make these extra assumptions beyond what's |
I consider the binary bitwise operations, for negative ints, to be
either undefined or wrongly implemented. Consider the following (3.1)
>>> 3^2
1
>>> -3^2
-1
>>> 3^-2
-3
>>> -3^-2
3
>>> 2^3
1
>>> -2^3
-3 Something change sign just flips the sign of the result, sometimes it The doc says only "The ^ operator yields the bitwise XOR (exclusive OR) My impression is that Python longs are signed magnitudes. If so, the |
Terry, the language reference also says: """ That explains every result you saw: 3 = ...000011 -3 = ...111101 3 = ...000011 -3 = ...111101 2 = ...000010 -2 = ...111110 In every case, the result is simply the xor of the inputs viewed as It's true that CPython's longs are /implemented/ via sign-magnitude, but |
Thanks Tim. I see that is back in 3.2 rather than in the shift and mask |
It looks like the fast paths for INPLACE_ADD and INPLACE_SUBTRACT in Python 2 don't have the cast-to-unsigned fix, so they're still relying on undefined behavior. For example, in INPLACE_ADD:
|
Python 2 is no longer supported. Python 3's _PyLong_Add() function doesn't rely on overflow. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: