Author gdr@garethrees.org
Recipients davin, dtasev, gdr@garethrees.org, sbt, serhiy.storchaka
Date 2017-07-13.15:59:55
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1499961596.11.0.572897975158.issue30919@psf.upfronthosting.co.za>
In-reply-to
Content
In Python 2.7, multiprocessing.heap.Arena uses an anonymous memory mapping on Unix. Anonymous memory mappings can be shared between processes but only via fork().

But Python 3 supports other ways of starting subprocesses (see issue 8713 [1]) and so an anonymous memory mapping no longer works. So instead a temporary file is created, filled with zeros to the given size, and mapped into memory (see changeset 3b82e0d83bf9 [2]). It is the zero-filling of the temporary file that takes the time, because this forces the operating system to allocate space on the disk.

But why not use ftruncate() (instead of write()) to quickly create a file with holes? POSIX says [3], "If the file size is increased, the extended area shall appear as if it were zero-filled" which would seem to satisfy the requirement.

[1] https://bugs.python.org/issue8713
[2] https://hg.python.org/cpython/rev/3b82e0d83bf9
[3] http://pubs.opengroup.org/onlinepubs/9699919799/functions/ftruncate.html
History
Date User Action Args
2017-07-13 15:59:56gdr@garethrees.orgsetrecipients: + gdr@garethrees.org, sbt, serhiy.storchaka, davin, dtasev
2017-07-13 15:59:56gdr@garethrees.orgsetmessageid: <1499961596.11.0.572897975158.issue30919@psf.upfronthosting.co.za>
2017-07-13 15:59:56gdr@garethrees.orglinkissue30919 messages
2017-07-13 15:59:55gdr@garethrees.orgcreate