This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author neologix
Recipients mboquien, neologix, pitrou, sbt
Date 2014-04-03.18:38:00
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <CAH_1eM0JMNSk2+OYkH7s-q_stMUCFSmPH8hXCYfO=YhMZjz0tg@mail.gmail.com>
In-reply-to <1396480853.06.0.555216825765.issue21116@psf.upfronthosting.co.za>
Content
> Also, the FreeBSD man page for mmap() has the following warning:

That's mostly important for real file-backed mapping.
In our case, we don't want a file-backed mmap: we expect the mapping to fit
entirely in memory, so the writeback/read performance isn't that important
to us.

> Using truncate() to zero extend is not really portable: it is only
guaranteed on XSI-compliant POSIX systems.

Now that's annoying.
How about trying file.truncate() within a try block, and if an error is
raised fallback to the zero-filling?

Doing a lot of IO for an object which is supposed to be used for shared
memory is sad.

Or maybe it's time to add an API to access shared memory from Python (since
that's really what we're trying to achieve here).

> According to the documentation, the returned shared array is zeroed.
> In that case because the entire array is written at allocation, the
process is expected to get killed
> if allocating more memory than available. Unless I am misunderstanding
something, which is entirely
> possible.

Having the memory zero-filed doesn't require a write at all: when you do an
anonymous memory mapping for let's say 1Gb, the kernel doesn't
pre-emptively zero-fill it, it would be way to slow: usually it just sets
up the process page table to make this area a COW of a single zero page:
upon read, you'll read zeros, and upon write, it'll duplicate it as needed.

The only reason the code currently zero-fills the file is to avoid the
portability issues detailed by Richard.
History
Date User Action Args
2014-04-03 18:38:00neologixsetrecipients: + neologix, pitrou, sbt, mboquien
2014-04-03 18:38:00neologixlinkissue21116 messages
2014-04-03 18:38:00neologixcreate