This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: anonymous mmap
Type: Stage:
Components: Library (Lib) Versions: Python 2.4
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: nnorwitz Nosy List: nnorwitz, titty
Priority: normal Keywords: patch

Created on 2006-01-16 08:44 by titty, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
patch titty, 2006-01-16 08:44 patch agains 2.4 maintenance branch
patch2.diff titty, 2006-01-18 09:16 patch handling /dev/zero, undefined MAP_ANON + tests
Messages (9)
msg49319 - (view) Author: Ralf Schmitt (titty) Date: 2006-01-16 08:44
make mmap.mmap(-1, size) map anonymous memory on both windows 
and unix-like systems.

If this goes in, I'll also write some documentation.

here's some more reasoning:
https://sourceforge.net/tracker/?
func=detail&atid=105470&aid=1402308&group_id=5470
msg49320 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2006-01-17 05:22
Logged In: YES 
user_id=33168

In addition to docs, tests will also be necessary before
committing.

What if fd == -1 and MAP_ANON is not defined?  It seems the
state could be inconsistent.  Shouldn't an exception be
raised in this case?
msg49321 - (view) Author: Ralf Schmitt (titty) Date: 2006-01-17 08:48
Logged In: YES 
user_id=17929

quoting ubuntu's mmap man page:

MAP_ANONYMOUS
  The mapping is not backed by any file; the fd and  offset
 arguments  are ignored.  This flag in conjunction with
MAP_SHARED is implemented since Linux 2.4.

MAP_ANON
  Alias for MAP_ANONYMOUS. Deprecated.

---

On a glibc 2.1 system (once installed with a 2.2 kernel, now
running a 2.4 one) the manpage doesn't even mention
MAP_ANON(YMOUS).
The patch in it's current form will just fail on system's
not defining MAP_ANON (i.e. mmap will fail and an exception
will be raised).

However, according to Stevens 'Advanced Programming in the
Unix Environment', chapter 14.9, there are two ways to mmap
anonymous memory.
The first one works by using MAP_ANON (4.3+BSD),
and the second one works by opening /dev/zero and passing
that as a filedescriptor to mmap (SVR4).

So, I guess all BSD's and newer Linux Systems would work
with this version. Anyway I can send another patch handling
that case if you like?
msg49322 - (view) Author: Ralf Schmitt (titty) Date: 2006-01-17 08:59
Logged In: YES 
user_id=17929

Well, seems like one would also need some special handling for 
systems only defining one of MAP_ANON/MAP_ANONYOUS:

http://www.winehq.org/hypermail/wine-devel/2004/12/0064.html
msg49323 - (view) Author: Ralf Schmitt (titty) Date: 2006-01-17 10:34
Logged In: YES 
user_id=17929

This patch tries to mmap via /dev/zero if neither MAP_ANON
nor MAP_ANONYMOUS is defined and handles the case where only
MAP_ANONYMOUS is defined.

I've added some tests, and found that test_mmap executed
mmap(-1, ...), which probably would have crashed the tests
running on OS X. Aren't the tests being run before release
on all major platforms?
msg49324 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2006-01-18 06:24
Logged In: YES 
user_id=33168

To answer your question:  
> Aren't the tests being run before release on all major
platforms?

It's difficult to answer.  I think generally the answer is
yes, but it all depends on volunteers.  We are currently
running tests on every checkin: 
http://www.python.org/dev/buildbot/ for both HEAD and 2.4
branch.  BuildBot is a relatively recent addition though. 
On OS X 10.3, we aren't having a problem with mmap(). 
Various developers I know use OS X, not sure which versions
of OS X though.

--
General comments about the patch:  For any solution, we
should strive to make it available if possible (with reason)
on all platforms.  We should attempt to have all versions of
Python behave similarly.  (I'm not trying to imply you
aren't working in this direction, it seems like you are.)

I only notice one patch attached.  Have you been deleting
the others or did you forget to check the litle box?
msg49325 - (view) Author: Ralf Schmitt (titty) Date: 2006-01-18 09:16
Logged In: YES 
user_id=17929

Thanks for the insight.
And yes, I forgot to check that little box.
msg49326 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2006-02-05 06:01
Logged In: YES 
user_id=33168

I modified the patch some.  Removed include <fcntl.h> since
that didn't seem necessary.  Changed the #ifdef slightly. 
Modified comments.

I now understand that anonymous memory was allowed on Unix
before (in 2.3 and pre 2.4.2).  I backported the non-feature
enhancements of this patch (not dup()ing -1).  So 2.4.3
should work like 2.4.1 and earlier.  It would be great if
you can test out 2.5 and 2.4.3 from SVN.

Committed revision 42244.
Committed revision 42245. (2.4)


msg49327 - (view) Author: Ralf Schmitt (titty) Date: 2006-02-05 15:09
Logged In: YES 
user_id=17929

The manpages for open(2) on FreeBSD 4.11, Ubuntu 5.10 and
OpenBSD 3.8 all mention '#include <fcntl.h>' in their
synopsis, so I guess it might be a good idea to include it.
I ran Lib/test/test_mmap.py for python trunk on FreeBSD,
OpenBSD, Ubuntu and Suse8/amd64 with success.

That one comment should now read 'maybe define MAP_ANONYMOUS
in terms of MAP_ANON'.

However, I didn't test 2.4.3...
History
Date User Action Args
2022-04-11 14:56:15adminsetgithub: 42799
2006-01-16 08:44:19tittycreate