This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author benjamin.peterson
Recipients benjamin.peterson
Date 2016-06-04.05:18:44
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1465017525.1.0.917023415627.issue27211@psf.upfronthosting.co.za>
In-reply-to
Content
Guido Vranken on PSRT report:

Python-2.7.11/Modules/_io/iobase.c iobase_readline():

531         old_size = PyByteArray_GET_SIZE(buffer);
532         PyByteArray_Resize(buffer, old_size + PyBytes_GET_SIZE(b));
533         memcpy(PyByteArray_AS_STRING(buffer) + old_size,
534                PyBytes_AS_STRING(b), PyBytes_GET_SIZE(b));

PyByteArray_Resize() relies on realloc(), and thus can fail if there
is insufficient memory. However, the return value of
PyByteArray_Resize() is not checked in iobase_readline(). So if
PyByteArray_Resize() fails, the memory area is too small for the data
that memcpy will write.

Proof Of Concept
----------------
First create a file with very long lines:

with open('out.xxx', 'wb') as fp:
    fp.write('x' * (1024*1024*64))
    fp.write('\n')

Zip it:

$ zip out.xxx.zip out.xxx

Set a smallish memory limit

$ ulimit -Sv 70000

Then run:

import zipfile
zf = zipfile.ZipFile('out.xxx.zip', 'r')
f = zf.open('out.xxx')
r = "x" * (1024*1024*20)
f.readline()
r = ""

----------------

For me this configuration resulted in

*** Error in `./python': realloc(): invalid next size: 0x00000000024b2f20 ***
Aborted

You might need to experiment a bit with the parameters (length of
lines, size of 'r', ulimit) to get a similar result. But there is a
bug there.

Let me know if you need more information from me.
History
Date User Action Args
2016-06-04 05:18:45benjamin.petersonsetrecipients: + benjamin.peterson
2016-06-04 05:18:45benjamin.petersonsetmessageid: <1465017525.1.0.917023415627.issue27211@psf.upfronthosting.co.za>
2016-06-04 05:18:44benjamin.petersonlinkissue27211 messages
2016-06-04 05:18:44benjamin.petersoncreate