This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: integer overflow in the _csv module's join_append_data function
Type: security Stage: resolved
Components: Library (Lib) Versions: Python 3.6, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 2.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, python-dev, tehybel
Priority: normal Keywords:

Created on 2016-08-14 00:16 by benjamin.peterson, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (3)
msg272624 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2016-08-14 00:16
Thomas E Hybel on PSRT reports:

This vulnerability is an integer overflow leading to a heap buffer overflow. I
have attached a proof-of-concept script below.

The vulnerability resides in the Modules/_csv.c file, in the join_append and
join_append_data functions.

join_append initially calls join_append_data with copy_phase=0 to compute the
new length of its internal "rec" buffer. Then it grows the buffer. Finally it
calls join_append_data with copy_phase=1 to perform the actual writing.

The root issue is that join_append_data does not check for overflow when
computing the field rec_len which it returns. By having join_append_data called
on a few fields of appropriate length, we can make rec_len roll around and
become a small integer.

Note that there is already a check in join_append for whether (rec_len < 0). But
this check is insufficient as we can cause rec_len to grow sufficiently in a
single call to never let join_append see a negative size.

After the overflow happens, rec_len is a small integer, and thus when
join_append calls join_check_rec_size to potentially grow the rec buffer, no
enlargement happens. After this, join_append_data is called again, now with
copy_phase=1, and with a giant field_len.

Thus join_append_data writes the remaining data out-of-bounds of the self->rec
buffer which is located on the heap. Such a complete heap corruption should
definitely be exploitable to gain remote code execution.

Further details:

Tested version: Python-3.5.2, 32 bits

Proof-of-concept reproducer script (32-bits only):

--- begin script ---

import _csv

class MockFile:
    def write(self, _):

writer = _csv.writer(MockFile())
writer.writerow(["A"*0x10000, '"'*0x7fffff00]) 

--- end script ---

Python (configured with --with-pydebug) segfaults when the script is run. A
backtrace can be seen below. Note that the script only crashes on 32-bit
versions of Python. That's because the rec_len variable is an ssize_t, which is
4 bytes wide on 32-bit architectures, but 8 bytes wide on 64-bit arches.

(gdb) r
Starting program: /home/ubuntu32/python3/Python-3.5.2/python ../
Program received signal SIGSEGV, Segmentation fault.
PyType_IsSubtype (a=0x0, b=b@entry=0x82d9aa0 <PyModule_Type>) at Objects/typeobject.c:1343
1343        mro = a->tp_mro;
(gdb) bt
#0  PyType_IsSubtype (a=0x0, b=b@entry=0x82d9aa0 <PyModule_Type>) at Objects/typeobject.c:1343
#1  0x080e29d9 in PyModule_GetState (m=0xb7c377f4) at Objects/moduleobject.c:532
#2  0xb7fd1a33 in join_append_data (self=self@entry=0xb7c2ffac, field_kind=field_kind@entry=0x1, field_data=field_data@entry=0x37c2f038,
    field_len=field_len@entry=0x7fffff00, quoted=quoted@entry=0xbffff710, copy_phase=copy_phase@entry=0x1)
    at /home/ubuntu32/python3/Python-3.5.2/Modules/_csv.c:1060
#3  0xb7fd1d6e in join_append (self=self@entry=0xb7c2ffac, field=field@entry=0x37c2f018, quoted=0x1, quoted@entry=0x0)
    at /home/ubuntu32/python3/Python-3.5.2/Modules/_csv.c:1138
msg272625 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-08-14 00:22
New changeset fdae903db33a by Benjamin Peterson in branch '2.7':
check for overflow in join_append_data (closes #27758)

New changeset afa356402217 by Benjamin Peterson in branch '3.3':
check for overflow in join_append_data (closes #27758)

New changeset 10b89df93c58 by Benjamin Peterson in branch '3.4':
merge 3.3 (#27758)

New changeset 55e8d3e542bd by Benjamin Peterson in branch '3.5':
merge 3.4 (closes #27758)

New changeset 609b554dd4a2 by Benjamin Peterson in branch 'default':
merge 3.5 (closes #27758)
msg272660 - (view) Author: tehybel (tehybel) Date: 2016-08-14 09:43
Thanks for fixing this. I looked at the patch and it seems correct.
Date User Action Args
2022-04-11 14:58:34adminsetgithub: 71945
2016-08-14 09:43:36tehybelsetnosy: + tehybel
messages: + msg272660
2016-08-14 00:22:42python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg272625

resolution: fixed
stage: resolved
2016-08-14 00:16:14benjamin.petersoncreate