classification
Title: Heap corruption via Python 2.7.11 IOBase readline()
Type: security Stage: resolved
Components: Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, python-dev, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2016-06-04 05:18 by benjamin.peterson, last changed 2016-06-04 05:31 by serhiy.storchaka. This issue is now closed.

Messages (3)
msg267231 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2016-06-04 05:18
Guido Vranken on PSRT report:

Python-2.7.11/Modules/_io/iobase.c iobase_readline():

531         old_size = PyByteArray_GET_SIZE(buffer);
532         PyByteArray_Resize(buffer, old_size + PyBytes_GET_SIZE(b));
533         memcpy(PyByteArray_AS_STRING(buffer) + old_size,
534                PyBytes_AS_STRING(b), PyBytes_GET_SIZE(b));

PyByteArray_Resize() relies on realloc(), and thus can fail if there
is insufficient memory. However, the return value of
PyByteArray_Resize() is not checked in iobase_readline(). So if
PyByteArray_Resize() fails, the memory area is too small for the data
that memcpy will write.

Proof Of Concept
----------------
First create a file with very long lines:

with open('out.xxx', 'wb') as fp:
    fp.write('x' * (1024*1024*64))
    fp.write('\n')

Zip it:

$ zip out.xxx.zip out.xxx

Set a smallish memory limit

$ ulimit -Sv 70000

Then run:

import zipfile
zf = zipfile.ZipFile('out.xxx.zip', 'r')
f = zf.open('out.xxx')
r = "x" * (1024*1024*20)
f.readline()
r = ""

----------------

For me this configuration resulted in

*** Error in `./python': realloc(): invalid next size: 0x00000000024b2f20 ***
Aborted

You might need to experiment a bit with the parameters (length of
lines, size of 'r', ulimit) to get a similar result. But there is a
bug there.

Let me know if you need more information from me.
msg267233 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-06-04 05:22
New changeset afbd4473947a by Benjamin Peterson in branch '2.7':
check the result of PyByteArray_Resize in readline() (closes #27211)
https://hg.python.org/cpython/rev/afbd4473947a
msg267234 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-06-04 05:31
This bug was fixed for 3.x in issue18408. Maybe we should backport other Victor's fixes from this issue.

Yet one possible bug -- using memcpy instead of memmove. The memory block can be resized without changing the address.
History
Date User Action Args
2016-06-04 05:31:23serhiy.storchakasetnosy: + vstinner, serhiy.storchaka
messages: + msg267234
2016-06-04 05:22:23python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg267233

resolution: fixed
stage: resolved
2016-06-04 05:18:45benjamin.petersoncreate