Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[easy C] array.array.index() method downcasts Py_ssize_t to long #85257

Closed
WildCard65 mannequin opened this issue Jun 23, 2020 · 9 comments
Closed

[easy C] array.array.index() method downcasts Py_ssize_t to long #85257

WildCard65 mannequin opened this issue Jun 23, 2020 · 9 comments
Labels
3.8 only security fixes 3.9 only security fixes 3.10 only security fixes easy stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@WildCard65
Copy link
Mannequin

WildCard65 mannequin commented Jun 23, 2020

BPO 41085
Nosy @pfmoore, @vstinner, @tjguk, @zware, @zooba, @WildCard65, @miss-islington
PRs
  • bpo-41085: Fixed 'array.array.index' downcasting 'Py_ssize_t' to 'long'. #21071
  • [3.9] bpo-41085: Fix array.array.index() on 64-bit Windows (GH-21071) #21076
  • [3.8] bpo-41085: Fix array.array.index() on 64-bit Windows (GH-21071) #21077
  • Files
  • test_output.log: Log of tests that were ran at time of upload.
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-06-23.13:52:54.522>
    created_at = <Date 2020-06-23.00:44:44.317>
    labels = ['easy', 'type-bug', '3.8', '3.9', '3.10', 'library']
    title = '[easy C] array.array.index() method downcasts Py_ssize_t to long'
    updated_at = <Date 2020-06-23.14:37:17.548>
    user = 'https://github.com/WildCard65'

    bugs.python.org fields:

    activity = <Date 2020-06-23.14:37:17.548>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-06-23.13:52:54.522>
    closer = 'WildCard65'
    components = ['Library (Lib)']
    creation = <Date 2020-06-23.00:44:44.317>
    creator = 'WildCard65'
    dependencies = []
    files = ['49257']
    hgrepos = []
    issue_num = 41085
    keywords = ['patch', 'easy (C)', 'newcomer friendly']
    message_count = 9.0
    messages = ['372135', '372140', '372153', '372155', '372156', '372164', '372166', '372167', '372175']
    nosy_count = 8.0
    nosy_names = ['paul.moore', 'vstinner', 'tim.golden', 'SilentGhost', 'zach.ware', 'steve.dower', 'WildCard65', 'miss-islington']
    pr_nums = ['21071', '21076', '21077']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue41085'
    versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    @WildCard65
    Copy link
    Mannequin Author

    WildCard65 mannequin commented Jun 23, 2020

    Here's the verbose stack trace of the failing test:

    ======================================================================
    FAIL: test_index (test.test_array.LargeArrayTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "L:\GIT\cpython\lib\test\support\__init__.py", line 837, in wrapper
        return f(self, maxsize)
      File "L:\GIT\cpython\lib\test\test_array.py", line 1460, in test_index
        self.assertEqual(example.index(11), size+3)
    AssertionError: -2147483645 != 2147483651

    @WildCard65 WildCard65 mannequin added tests Tests in the Lib/test dir 3.10 only security fixes type-bug An unexpected behavior, bug, or error labels Jun 23, 2020
    @SilentGhost
    Copy link
    Mannequin

    SilentGhost mannequin commented Jun 23, 2020

    This looks like an overflow failure in a bigmem test (though the given limit 24Gb should accommodate that?). Do bigmem tests work on Windows?

    William, you've made some sort of modifications to the versioned files, what are those?

    @SilentGhost SilentGhost mannequin added OS-windows labels Jun 23, 2020
    @WildCard65
    Copy link
    Mannequin Author

    WildCard65 mannequin commented Jun 23, 2020

    The only modification I made was to "rt.bat" to have the value of '-u' work properly.

    @vstinner
    Copy link
    Member

    It's Python 3.10 on Windows built in 64-bit with Visual Studio.

    Extract of test_output.log:

    L:\GIT\cpython\PCbuild>"L:\GIT\cpython\PCbuild\amd64\python_d.exe" -u -Wd -E -bb -m test -u all,-curses -v -M 24Gb --header

    == CPython 3.10.0a0 (heads/master-dirty:c96d00e88e, Jun 22 2020, 19:15:48) [MSC v.1926 64 bit (AMD64)]
    == Windows-10-10.0.19041-SP0 little-endian
    == cwd: L:\GIT\cpython\build\test_python_20756
    == CPU count: 8
    == encodings: locale=cp1252, FS=utf-8
    (...)
    test_index (test.test_array.LargeArrayTest) ...
    ... expected peak memory use: 4.2G
    FAIL
    (...)
    ======================================================================
    FAIL: test_index (test.test_array.LargeArrayTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "L:\GIT\cpython\lib\test\support\__init__.py", line 837, in wrapper
        return f(self, maxsize)
      File "L:\GIT\cpython\lib\test\test_array.py", line 1460, in test_index
        self.assertEqual(example.index(11), size+3)
    AssertionError: -2147483645 != 2147483651

    @vstinner
    Copy link
    Member

    That's a bug in array_array_index() which downcasts "Py_ssize_t i" to long:

    static PyObject *
    array_array_index(arrayobject *self, PyObject *v)
    /*[clinic end generated code: output=d48498d325602167 input=cf619898c6649d08]*/
    {
        Py_ssize_t i;
    
        for (i = 0; i < Py_SIZE(self); i++) {
            PyObject *selfi;
            int cmp;
    
            selfi = getarrayitem((PyObject *)self, i);
            if (selfi == NULL)
                return NULL;
            cmp = PyObject_RichCompareBool(selfi, v, Py_EQ);
            Py_DECREF(selfi);
            if (cmp > 0) {
                return PyLong_FromLong((long)i); // <===== HERE ===
            }
            else if (cmp < 0)
                return NULL;
        }
        PyErr_SetString(PyExc_ValueError, "array.index(x): x not in array");
        return NULL;
    }

    @vstinner vstinner added the easy label Jun 23, 2020
    @vstinner vstinner changed the title Array regression test fails [easy C] Array regression test fails Jun 23, 2020
    @vstinner vstinner added the easy label Jun 23, 2020
    @vstinner vstinner changed the title Array regression test fails [easy C] Array regression test fails Jun 23, 2020
    @vstinner vstinner changed the title [easy C] Array regression test fails [easy C] array.array.index() method downcasts Py_ssize_t to long Jun 23, 2020
    @vstinner vstinner changed the title [easy C] Array regression test fails [easy C] array.array.index() method downcasts Py_ssize_t to long Jun 23, 2020
    @vstinner
    Copy link
    Member

    New changeset 1d3dad5 by WildCard65 in branch 'master':
    bpo-41085: Fix array.array.index() on 64-bit Windows (GH-21071)
    1d3dad5

    @miss-islington
    Copy link
    Contributor

    New changeset c6e24e7 by Miss Islington (bot) in branch '3.8':
    bpo-41085: Fix array.array.index() on 64-bit Windows (GH-21071)
    c6e24e7

    @miss-islington
    Copy link
    Contributor

    New changeset 92f8b48 by Miss Islington (bot) in branch '3.9':
    bpo-41085: Fix array.array.index() on 64-bit Windows (GH-21071)
    92f8b48

    @WildCard65 WildCard65 mannequin closed this as completed Jun 23, 2020
    @WildCard65 WildCard65 mannequin closed this as completed Jun 23, 2020
    @vstinner
    Copy link
    Member

    Thanks William Pickard for the bug report and the fix!

    @vstinner vstinner added stdlib Python modules in the Lib dir and removed tests Tests in the Lib/test dir OS-windows labels Jun 23, 2020
    @vstinner vstinner added 3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir and removed tests Tests in the Lib/test dir OS-windows labels Jun 23, 2020
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes 3.9 only security fixes 3.10 only security fixes easy stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants