Message126104
The UNIX mmap version in
Modules/mmapmodule.c
makes a call to dup() to duplicate the file descriptor that is passed for creating the memory-mapped region.
This way, I frequently hit the limit for the number of open file handles while creating mmap.mmap() instances, despite closing all my opened files after creating a mapping for them.
My application is scientific data (read: "large data" :-) analysis for which mmap() is very well suited together with numpy/scipy - however, the large number of files causes me to hit the resource limit on open file handles)
I propose to remove this dup(), which was apparently introduced in the process of fixing issue #728515, concerning incorrect behavior of the mmap.resize() method on UNIX, as it was feared the user could have closed the file handle already.
I think it should be the responsibility of the user not to close the file in question or either - if it needs to be closed - not to call the resize method later.
I should stress that a call to mmap(2) does not require an open file handle during the time of the mapping but that a duplicate of the file handle was only kept to allow .size() and .resize() to work. See the POSIX Programmer's Manual:
The mmap() function shall add an extra reference to the file
associated with the file descriptor fildes which is not removed
by a subsequent close() on that file descriptor. This reference
shall be removed when there are no more mappings to the file.
It would be great if this little nicety would translate better to Python!
Regards,
Lorenz |
|
Date |
User |
Action |
Args |
2011-01-12 15:16:19 | lorenz | set | recipients:
+ lorenz |
2011-01-12 15:16:19 | lorenz | set | messageid: <1294845379.76.0.427887707002.issue10897@psf.upfronthosting.co.za> |
2011-01-12 15:16:15 | lorenz | link | issue10897 messages |
2011-01-12 15:16:14 | lorenz | create | |
|