# 3617                for (i = 0; i < len; i++) { 
# (gdb) print *(PyListObject*)tmp
# $1 = {ob_base = {ob_base = {_ob_next = 0x4056f8f4, _ob_prev = 0x4057329c, ob_refcnt = 2, ob_type = 0x830e1c0 <PyList_Type>}, 
#     ob_size = 1337}, ob_item = 0x8491ae0, allocated = 1432}
# (gdb) n
# 3619                    res = obj2ast_stmt(PyList_GET_ITEM(tmp, i), &value, arena);
# (gdb) n
# 3620                    if (res != 0) goto failed;
# (gdb) print *(PyListObject*)tmp
# $2 = {ob_base = {ob_base = {_ob_next = 0x4056f8f4, _ob_prev = 0x4057329c, ob_refcnt = 2, ob_type = 0x830e1c0 <PyList_Type>}, 
#     ob_size = 1}, ob_item = 0x8491ae0, allocated = 4}
# (gdb) c
# Continuing.
# 
# Program received signal SIGSEGV, Segmentation fault.
# 0x080f2c17 in PyObject_GetAttr (v=<unknown at remote 0x405733b4>, name='lineno') at Objects/object.c:872
# 872         if (tp->tp_getattro != NULL)
# 
# Objects freed in __getattr__ are used later in the loop above. There are two
# bugs actually. One is the use-after-free and the second is using a stale size
# variable "len" to control the for(...) loop. "body" can be mutated inside
# obj2ast_stmt.


This construct:

            for (i = 0; i < len; i++) {
                stmt_ty value;
                res = obj2ast_stmt(PyList_GET_ITEM(tmp, i), &value, arena);
                if (res != 0) goto failed;
                asdl_seq_SET(body, i, value);
            }

is repeated multiple times in multiple obj2ast_ methods. It contains two bugs:
1. tmp[i] isn't protected from deletion inside python code (refcnt is not increased by GET_ITEM),
2. tmp's length can drop below "len" resulting in an OOB read, because the loop counter is static.

ping

ping

ping

ping

Fix by replacing static 'len' in loops with a macro, so that mutations of   size of the containter do not casue OOB reads.

Please note that Python/Python-ast.c is automatically generated by Parser/asdl_c.py.

Bad things happen not only when a list shrinks, but also when it grows during iteration.

The one solution is to check if the size is changed on every iteration. The other solution is to convert a list to a tuple for iterating.

New changeset 47d5bf5a846f by Serhiy Storchaka in branch '2.7':
Issue #24098: Fixed possible crash when AST is changed in process of
https://hg.python.org/cpython/rev/47d5bf5a846f

New changeset f575710b5f56 by Serhiy Storchaka in branch '3.5':
Issue #24098: Fixed possible crash when AST is changed in process of
https://hg.python.org/cpython/rev/f575710b5f56

New changeset 7528154cadaa by Serhiy Storchaka in branch '3.6':
Issue #24098: Fixed possible crash when AST is changed in process of
https://hg.python.org/cpython/rev/7528154cadaa

New changeset def217aaad2f by Serhiy Storchaka in branch 'default':
Issue #24098: Fixed possible crash when AST is changed in process of
https://hg.python.org/cpython/rev/def217aaad2f