Message78076
Notes on the problem.
The Python mmap documentation says: If length is 0, the maximum length
of the map will be the current size of the file when mmap is called.
Currently, on a system where off_t is 64-bit and size_t is 32-bit, if
the size of the file (st_size) is > 0xFFFF'FFFF, the code will in effect
try to map (st_size & 0xFFFF'FFFF). This is suboptimal (imagine if the
size of the file is 0x1'0000'1000 or even 0x1'0000'0000).
In addition, it seems weird that a caller would have to check the size
of the returned mmap against the size of the file to see if the entire
file was mapped or not.
Finally, it appears that there isn't a reliable, architecture
independent, quick, and easy way to figure out what the maximum mmap
size is.
With these points in mind, I am going to work on coming up with a patch
that will change the behavior to simply return an error when a zero
length is passed to mmap and the entire file can not be mapped. This
will put the onus on the caller to determine an appropriate mmap size
for "chunking" the processing of the file, but the behavior will become
more consistent.
Of course, comments and suggestions are welcome. However, I'm going to
have to wrap up my immediate involvement on this in the next couple of
days (Yay, vacation!). |
|
Date |
User |
Action |
Args |
2008-12-19 23:24:11 | saa | set | recipients:
+ saa, loewis |
2008-12-19 23:24:11 | saa | set | messageid: <1229729051.64.0.809371586198.issue4681@psf.upfronthosting.co.za> |
2008-12-19 23:24:10 | saa | link | issue4681 messages |
2008-12-19 23:24:09 | saa | create | |
|