classification
Title: integer overflow in the size of a ctypes.Array
Type: behavior Stage: resolved
Components: ctypes Versions: Python 3.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Do not assume signed integer overflow behavior
View: 1621
Assigned To: Nosy List: Oren Milman, serhiy.storchaka
Priority: normal Keywords:

Created on 2017-09-29 10:39 by Oren Milman, last changed 2017-09-29 13:33 by serhiy.storchaka. This issue is now closed.

Messages (3)
msg303322 - (view) Author: Oren Milman (Oren Milman) * Date: 2017-09-29 10:39
The following code:
from ctypes import *
from _testcapi import PY_SSIZE_T_MAX, LONG_MAX

if LONG_MAX == PY_SSIZE_T_MAX == (1 << 31) - 1:
    class MyArray(Array):
        _type_ = c_longlong
        _length_ = 1 << 29

    arr = MyArray()
    for i in range(3):
        arr[i] = i
    for i in range(3):
        print(arr[i])

Produces this output (on a 32bit Python on my Windows 10):
2
2
2

This is because PyCArrayType_new() (in Modules/_ctypes/_ctypes.c) raises a
"array too large" error in case (length * itemsize < 0). However, this
multiplication might also overflow to a non-negative number, e.g. to zero in
the code above.
PyCArrayType_new() then does:
    stgdict->size = itemsize * length;

Array_ass_item() and Array_item() both do:
    size = stgdict->size / stgdict->length;
    offset = index * size;

So in the above code, the integer overflow caused the array to collapse to a
single element (the first element).


ISTM that we can fix this by changing the overflow detection logic to this:
    assert(itemsize >= 0 && length >= 0);
    array_size = itemsize * length;
    if (itemsize && array_size / itemsize != length) {
        PyErr_SetString(PyExc_OverflowError,
                        "array too large");
        goto error;
    }

The assertion is guaranteed to be true after #29843 is resolved. (I would open
a PR for #29843 soon.)
msg303324 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-09-29 11:19
Isn't this a duplicate of issue1621?
msg303326 - (view) Author: Oren Milman (Oren Milman) * Date: 2017-09-29 11:50
oh, i missed that. sorry.
History
Date User Action Args
2017-09-29 13:33:05serhiy.storchakasetsuperseder: Do not assume signed integer overflow behavior
2017-09-29 11:50:34Oren Milmansetstatus: open -> closed
resolution: duplicate
stage: resolved
2017-09-29 11:50:12Oren Milmansetmessages: + msg303326
2017-09-29 11:19:19serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg303324
2017-09-29 10:39:58Oren Milmancreate