This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: chain.__setstate__ Type Confusion
Type: crash Stage: resolved
Components: Library (Lib) Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: JohnLeitch, python-dev, rhettinger, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2016-10-01 06:27 by JohnLeitch, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
itertools.patch JohnLeitch, 2016-10-01 06:27 Patch review
Py35_itertools.py JohnLeitch, 2016-10-01 06:29 Repro file
itertools_setstate.patch serhiy.storchaka, 2016-10-01 10:06 review
Pull Requests
URL Status Linked Edit
PR 552 closed dstufft, 2017-03-31 16:36
Messages (5)
msg277799 - (view) Author: John Leitch (JohnLeitch) * Date: 2016-10-01 06:27
Python 3.5.2 suffers from a type confusion vulnerability in the chain.__setstate__ method of the itertools module. The issue exists due to lack of argument validation in the chain_setstate() function:

static PyObject *
chain_setstate(chainobject *lz, PyObject *state)
{
    PyObject *source, *active=NULL;

    if (! PyArg_ParseTuple(state, "O|O", &source, &active))
        return NULL;

    Py_INCREF(source);
    Py_XSETREF(lz->source, source);
    Py_XINCREF(active);
    Py_XSETREF(lz->active, active);
    Py_RETURN_NONE;
}

After parsing the argument tuple, source and active are set without validating that they are iterator objects. This causes issues elsewhere, where the values are passed PyIter_Next:

static PyObject *
chain_next(chainobject *lz)
{
    PyObject *item;

    if (lz->source == NULL)
        return NULL;                                    /* already stopped */

    if (lz->active == NULL) {
        PyObject *iterable = PyIter_Next(lz->source);
        if (iterable == NULL) {
            Py_CLEAR(lz->source);
            return NULL;                                /* no more input sources */
        }
        lz->active = PyObject_GetIter(iterable);
        Py_DECREF(iterable);
        if (lz->active == NULL) {
            Py_CLEAR(lz->source);
            return NULL;                                /* input not iterable */
        }
    }
    item = PyIter_Next(lz->active);
    if (item != NULL)
        return item;
    if (PyErr_Occurred()) {
        if (PyErr_ExceptionMatches(PyExc_StopIteration))
            PyErr_Clear();
        else
            return NULL;                                /* input raised an exception */
    }
    Py_CLEAR(lz->active);
    return chain_next(lz);                      /* recurse and use next active */
}

In some cases, this can lead to a DEP access violation. It might be possible to exploit this to achieve code execution.

(4074.198c): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=00000000 ebx=0132fa10 ecx=5b547028 edx=00000002 esi=0132fa10 edi=5b37b3e0
eip=00000000 esp=009ef940 ebp=009ef94c iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
00000000 ??              ???
0:000> k6
ChildEBP RetAddr  
WARNING: Frame IP not in any known module. Following frames may be wrong.
009ef93c 5b329ac0 0x0
009ef94c 5b2cb321 python35!PyIter_Next+0x10 [c:\build\cpython\objects\abstract.c @ 2778]
009ef960 5b37b42e python35!chain_next+0x21 [c:\build\cpython\modules\itertoolsmodule.c @ 1846]
009ef970 5b33fedd python35!wrap_next+0x4e [c:\build\cpython\objects\typeobject.c @ 5470]
009ef990 5b328b9d python35!wrapper_call+0x7d [c:\build\cpython\objects\descrobject.c @ 1195]
009ef9ac 5b3c463c python35!PyObject_Call+0x6d [c:\build\cpython\objects\abstract.c @ 2167]

To fix this issue, it is recommended that chain_setstate() be updated to validate its arguments. A proposed patch has been attached.

static PyObject *
chain_setstate(chainobject *lz, PyObject *state)
{
    PyObject *source, *active=NULL;

    if (! PyArg_ParseTuple(state, "O|O", &source, &active))
        return NULL;

    if (!PyIter_Check(source) || (active != NULL && !PyIter_Check(active))) {
        PyErr_SetString(PyExc_ValueError, "Arguments must be iterators.");
        return NULL;
    }

    Py_INCREF(source);
    Py_XSETREF(lz->source, source);
    Py_XINCREF(active);
    Py_XSETREF(lz->active, active);
    Py_RETURN_NONE;
}
msg277804 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-01 10:06
There is similar issue with itertools.cycle(). And leaking SystemError to user code should be considered as a bug.

Proposed patch fixes these issues.
msg277848 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-10-02 04:25
The patch looks reasonable.
msg277855 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-10-02 06:17
New changeset 258ebc539b2e by Serhiy Storchaka in branch '3.5':
Issue #28322: Fixed possible crashes when unpickle itertools objects from
https://hg.python.org/cpython/rev/258ebc539b2e

New changeset c4937d066a8e by Serhiy Storchaka in branch '3.6':
Issue #28322: Fixed possible crashes when unpickle itertools objects from
https://hg.python.org/cpython/rev/c4937d066a8e

New changeset cb0755aa9f3d by Serhiy Storchaka in branch 'default':
Issue #28322: Fixed possible crashes when unpickle itertools objects from
https://hg.python.org/cpython/rev/cb0755aa9f3d
msg277856 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-02 06:21
Thank you for your contribution John.

cycle.__setstate__() was fixed in a4d5ef7fdec3 in 3.6, but the fix was not backported to 3.5. There is no pickle support in 2.7.
History
Date User Action Args
2022-04-11 14:58:37adminsetgithub: 72509
2017-03-31 16:36:22dstufftsetpull_requests: + pull_request964
2016-10-02 06:21:44serhiy.storchakasetstatus: open -> closed
versions: - Python 2.7
messages: + msg277856

assignee: serhiy.storchaka
resolution: fixed
stage: patch review -> resolved
2016-10-02 06:17:32python-devsetnosy: + python-dev
messages: + msg277855
2016-10-02 04:25:34rhettingersetmessages: + msg277848
2016-10-01 10:06:59serhiy.storchakasetfiles: + itertools_setstate.patch
type: security -> crash
messages: + msg277804
2016-10-01 06:34:45serhiy.storchakasetnosy: + rhettinger, serhiy.storchaka
stage: patch review

versions: + Python 2.7, Python 3.6, Python 3.7
2016-10-01 06:29:53JohnLeitchsetfiles: + Py35_itertools.py
2016-10-01 06:27:48JohnLeitchcreate