Author gdr@garethrees.org
Recipients davin, dtasev, gdr@garethrees.org, sbt, serhiy.storchaka
Date 2017-07-14.13:35:48
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1500039349.03.0.279441401792.issue30919@psf.upfronthosting.co.za>
In-reply-to
Content
I propose:

1. Ask Richard Oudkerk why in changeset 3b82e0d83bf9 the temporary file is zero-filled and not truncated. Perhaps there's some file system where this is necessary? (I tested HFS+ which doesn't support sparse files, and zero-filling seems not to be necessary, but maybe there's some other file system where it is?)

2. If there's no good reason for zero-filling the temporary file, replace it with a call to os.ftruncate(fd, size).

3. Update the documentation to mention the performance issue when porting multiprocessing code from 2 to 3. Unfortunately, I don't think there's any advice that the documentation can give that will help work around it -- monkey-patching works but is not supported.

4. Consider writing a fix, or at least a supported workaround. Here's a suggestion: update multiprocessing.sharedctypes and multiprocessing.heap so that they use anonymous maps in the 'fork' context. The idea is to update the RawArray and RawValue functions so that they take the context, and then pass the context down to _new_value, BufferWrapper.__init__ and thence to Heap.malloc where it can be used to determine what kind of Arena (file-backed or anonymous) should be used to satisfy the allocation request. The Heap class would have to have to segregate its blocks according to what kind of Arena they come from.
History
Date User Action Args
2017-07-14 13:35:49gdr@garethrees.orgsetrecipients: + gdr@garethrees.org, sbt, serhiy.storchaka, davin, dtasev
2017-07-14 13:35:49gdr@garethrees.orgsetmessageid: <1500039349.03.0.279441401792.issue30919@psf.upfronthosting.co.za>
2017-07-14 13:35:49gdr@garethrees.orglinkissue30919 messages
2017-07-14 13:35:48gdr@garethrees.orgcreate