Message70924
I wrote a buggy PNG parser which ended up doing several file.read(large
value). It causes a MemoryError, which was strange because the file was
only a few KB long.
I tracked it down to the implementation of read(). When given a size
hint it preallocates the return string with that size. If the hint is
for 10MB then the string returned will be preallocated fro 10MB, even if
the actual read is empty.
Here's a reproducible
BLOCKSIZE = 10*1024*1024
f=open("empty.txt", "w")
f.close()
f=open("empty.txt")
data = []
for i in range(10000):
s = f.read(BLOCKSIZE)
assert len(s) == 0
data.append(s)
I wasn't sure if this is properly a bug, but since the MemoryError
exception I got was quite unexpected and required digging into the
source code to figure out, I'll say that it is. |
|
Date |
User |
Action |
Args |
2008-08-09 02:29:45 | dalke | set | recipients:
+ dalke |
2008-08-09 02:29:45 | dalke | set | messageid: <1218248985.75.0.478147960205.issue3531@psf.upfronthosting.co.za> |
2008-08-09 02:29:43 | dalke | link | issue3531 messages |
2008-08-09 02:29:43 | dalke | create | |
|