This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author saa
Recipients loewis, saa
Date 2008-12-19.23:24:09
SpamBayes Score 6.395311e-06
Marked as misclassified No
Message-id <1229729051.64.0.809371586198.issue4681@psf.upfronthosting.co.za>
In-reply-to
Content
Notes on the problem.

The Python mmap documentation says: If length is 0, the maximum length 
of the map will be the current size of the file when mmap is called.

Currently, on a system where off_t is 64-bit and size_t is 32-bit, if 
the size of the file (st_size) is > 0xFFFF'FFFF, the code will in effect 
try to map (st_size & 0xFFFF'FFFF).  This is suboptimal (imagine if the 
size of the file is 0x1'0000'1000 or even 0x1'0000'0000).

In addition, it seems weird that a caller would have to check the size 
of the returned mmap against the size of the file to see if the entire 
file was mapped or not.

Finally, it appears that there isn't a reliable, architecture 
independent, quick, and easy way to figure out what the maximum mmap 
size is.

With these points in mind, I am going to work on coming up with a patch 
that will change the behavior to simply return an error when a zero 
length is passed to mmap and the entire file can not be mapped.  This 
will put the onus on the caller to determine an appropriate mmap size 
for "chunking" the processing of the file, but the behavior will become 
more consistent.

Of course, comments and suggestions are welcome.  However, I'm going to 
have to wrap up my immediate involvement on this in the next couple of 
days (Yay, vacation!).
History
Date User Action Args
2008-12-19 23:24:11saasetrecipients: + saa, loewis
2008-12-19 23:24:11saasetmessageid: <1229729051.64.0.809371586198.issue4681@psf.upfronthosting.co.za>
2008-12-19 23:24:10saalinkissue4681 messages
2008-12-19 23:24:09saacreate