classification
Title: growable_comment_array_add leaks, causes crash
Type: crash Stage: resolved
Components: Interpreter Core Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Alexander Riccio, benjamin.peterson, pablogsal
Priority: normal Keywords: patch

Created on 2020-03-20 00:58 by Alexander Riccio, last changed 2020-03-31 20:37 by Alexander Riccio. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19083 merged Alexander Riccio, 2020-03-20 01:48
Messages (5)
msg364644 - (view) Author: Alexander Riccio (Alexander Riccio) * Date: 2020-03-20 00:58
growable_comment_array_add in parsetok.c incorrectly uses realloc, which leaks the array when allocation fails, and then causes a null pointer deref crash later when the array is freed in growable_comment_array_deallocate (the array pointer is dereferenced, passing null to free is fine).

It's unlikely that this codepath is reached in normal use, since type comments need to be turned on (via the PyCF_TYPE_COMMENTS compiler flag), but I've managed to replicate the issue by injecting faults with Application Verifier. It's easiest to cause it to fail with a very large number of type comments, but presumably this could also happen with some form of heap fragmentation.

The buggy code is:

static int
growable_comment_array_add(growable_comment_array *arr, int lineno, char *comment) {
    if (arr->num_items >= arr->size) {
        arr->size *= 2;
        arr->items = realloc(arr->items, arr->size * sizeof(*arr->items));
        if (!arr->items) {
            return 0;
        }
    }

    arr->items[arr->num_items].lineno = lineno;
    arr->items[arr->num_items].comment = comment;
    arr->num_items++;
    return 1;
}


and the correct code would be something like:

static int
growable_comment_array_add(growable_comment_array *arr, int lineno, char *comment) {
    if (arr->num_items >= arr->size) {
        arr->size *= 2;
        void* new_items_array = realloc(arr->items, arr->size * sizeof(*arr->items));
        if (!new_items_array) {
            return 0;
        }
        arr->items = new_items_array;
    }

    arr->items[arr->num_items].lineno = lineno;
    arr->items[arr->num_items].comment = comment;
    arr->num_items++;
    return 1;
}
msg364645 - (view) Author: Alexander Riccio (Alexander Riccio) * Date: 2020-03-20 01:14
Sidenote: visual studio was misleading and made this look like a use-after-free for a little while, which was interesting.
msg365351 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-30 21:16
New changeset 51e3e450fbed46198d9be92add1a5dee6a1f7f41 by Alexander Riccio in branch 'master':
bpo-40020: Fix realloc leak on failure in growable_comment_array_add (GH-19083)
https://github.com/python/cpython/commit/51e3e450fbed46198d9be92add1a5dee6a1f7f41
msg365352 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-30 21:23
Alexander Riccio: Do you know want to propose a change to replace direct usage of malloc/realloc/free with PyMem_Malloc, PyMem_Realloc and PyMem_RawFree? It would add their builtin debug feature for free, and also detect most obvious buffer overflow (reject size larger than PY_SSIZE_T_MAX).
msg365421 - (view) Author: Alexander Riccio (Alexander Riccio) * Date: 2020-03-31 20:37
Sure, should I open a new issue?
History
Date User Action Args
2020-03-31 20:37:15Alexander Ricciosetstatus: open -> closed

nosy: - vstinner
messages: + msg365421

resolution: fixed
stage: patch review -> resolved
2020-03-30 21:23:29vstinnersetmessages: + msg365352
2020-03-30 21:16:06vstinnersetnosy: + vstinner
messages: + msg365351
2020-03-20 11:12:14corona10setversions: + Python 3.7, Python 3.8
2020-03-20 01:48:17Alexander Ricciosetkeywords: + patch
stage: patch review
pull_requests: + pull_request18442
2020-03-20 01:14:30Alexander Ricciosetnosy: + pablogsal
messages: + msg364645
2020-03-20 00:58:27Alexander Ricciocreate