This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author dalke
Recipients dalke
Date 2008-08-09.02:29:43
SpamBayes Score 0.000200912
Marked as misclassified No
Message-id <1218248985.75.0.478147960205.issue3531@psf.upfronthosting.co.za>
In-reply-to
Content
I wrote a buggy PNG parser which ended up doing several file.read(large 
value).  It causes a MemoryError, which was strange because the file was 
only a few KB long.

I tracked it down to the implementation of read().  When given a size 
hint it preallocates the return string with that size.  If the hint is 
for 10MB then the string returned will be preallocated fro 10MB, even if 
the actual read is empty.

Here's a reproducible

BLOCKSIZE = 10*1024*1024

f=open("empty.txt", "w")
f.close()

f=open("empty.txt")
data = []
for i in range(10000):
    s = f.read(BLOCKSIZE)
    assert len(s) == 0
    data.append(s)


I wasn't sure if this is properly a bug, but since the MemoryError 
exception I got was quite unexpected and required digging into the 
source code to figure out, I'll say that it is.
History
Date User Action Args
2008-08-09 02:29:45dalkesetrecipients: + dalke
2008-08-09 02:29:45dalkesetmessageid: <1218248985.75.0.478147960205.issue3531@psf.upfronthosting.co.za>
2008-08-09 02:29:43dalkelinkissue3531 messages
2008-08-09 02:29:43dalkecreate