\bpo-3617 for (i = 0; i < len; i++) {
# (gdb) print *(PyListObject*)tmp
# $1 = {ob_base = {ob_base = {ob_next = 0x4056f8f4, _ob_prev = 0x4057329c, ob_refcnt = 2, ob_type = 0x830e1c0 <PyList_Type>},
# ob_size = 1337}, ob_item = 0x8491ae0, allocated = 1432}
# (gdb) n
\bpo-3619 res = obj2ast_stmt(PyList_GET_ITEM(tmp, i), &value, arena);
# (gdb) n
\bpo-3620 if (res != 0) goto failed;
# (gdb) print *(PyListObject*)tmp
# $2 = {ob_base = {ob_base = {ob_next = 0x4056f8f4, _ob_prev = 0x4057329c, ob_refcnt = 2, ob_type = 0x830e1c0 <PyList_Type>},
# ob_size = 1}, ob_item = 0x8491ae0, allocated = 4}
# (gdb) c
# Continuing.
#
# Program received signal SIGSEGV, Segmentation fault.
# 0x080f2c17 in PyObject_GetAttr (v=<unknown at remote 0x405733b4>, name='lineno') at Objects/object.c:872
# 872 if (tp->tp_getattro != NULL)
#
# Objects freed in __getattr are used later in the loop above. There are two
# bugs actually. One is the use-after-free and the second is using a stale size
# variable "len" to control the for(...) loop. "body" can be mutated inside
# obj2ast_stmt.

This construct:

            for (i = 0; i < len; i++) {
                stmt_ty value;
                res = obj2ast_stmt(PyList_GET_ITEM(tmp, i), &value, arena);
                if (res != 0) goto failed;
                asdl_seq_SET(body, i, value);
            }

is repeated multiple times in multiple obj2ast_ methods. It contains two bugs:

1. tmp[i] isn't protected from deletion inside python code (refcnt is not increased by GET_ITEM),
2. tmp's length can drop below "len" resulting in an OOB read, because the loop counter is static.

ping

Fix by replacing static 'len' in loops with a macro, so that mutations of size of the containter do not casue OOB reads.

Please note that Python/Python-ast.c is automatically generated by Parser/asdl_c.py.

Bad things happen not only when a list shrinks, but also when it grows during iteration.

The one solution is to check if the size is changed on every iteration. The other solution is to convert a list to a tuple for iterating.

New changeset 47d5bf5a846f by Serhiy Storchaka in branch '2.7':
Issue bpo-24098: Fixed possible crash when AST is changed in process of
https://hg.python.org/cpython/rev/47d5bf5a846f

New changeset f575710b5f56 by Serhiy Storchaka in branch '3.5':
Issue bpo-24098: Fixed possible crash when AST is changed in process of
https://hg.python.org/cpython/rev/f575710b5f56

New changeset 7528154cadaa by Serhiy Storchaka in branch '3.6':
Issue bpo-24098: Fixed possible crash when AST is changed in process of
https://hg.python.org/cpython/rev/7528154cadaa

New changeset def217aaad2f by Serhiy Storchaka in branch 'default':
Issue bpo-24098: Fixed possible crash when AST is changed in process of
https://hg.python.org/cpython/rev/def217aaad2f