This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: mmap.mmap() allocates a file descriptor that isn't CLOEXEC
Type: resource usage Stage: needs patch
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Python 2.7: make private file descriptors non inheritable
View: 26769
Assigned To: Nosy List: Arfrever, christian.heimes, docs@python, martin.panter, neologix, rfm, vstinner
Priority: normal Keywords:

Created on 2013-10-30 05:19 by rfm, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (9)
msg201709 - (view) Author: Robert Merrill (rfm) Date: 2013-10-30 05:19
Same code but different problem as this issue: http://bugs.python.org/issue10897

The above-linked issue was closed as WONTFIX, but there is a secondary problem: the file descriptor that mmap.mmap() allocates is not set to close-on-exec.

This means that any process you spawn using os.system() and the like, its children, its children's children etc will inherit this file descriptor.

This is unexpected, unnecessary behavior, and potentially a huge pain if you mmap() a file and then spawn, say, inetd or something that does similar stuff, because now you have to kill every process that spawned from that in order to unmount filesystems or remove kernel modules that are being blocked as a result of the fds being open.
msg201710 - (view) Author: Robert Merrill (rfm) Date: 2013-10-30 05:25
I should add a caveat: the fd that is created by mmap /will/ be set to close-on-exec if the fd you passed in was. But even if it's not, I see no reason why mmap should not be setting it anyway.

At the very least, the documentation should bring the user's attention to this potential stumbling block. Currently it makes no mention of the fact that mmap duplicates file descriptors at all, much less how the flags are set.
msg201718 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-10-30 09:23
Can you suggest a documentation update?
msg201720 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-10-30 09:33
"the file descriptor that mmap.mmap() allocates is not set to close-on-exec"

In Python 3.4, the file descriptor is now non-inheritable, as a side effect of the PEP 446.
http://www.python.org/dev/peps/pep-0446/
msg201722 - (view) Author: Robert Merrill (rfm) Date: 2013-10-30 09:45
I'm adding Library again because I think the current behavior is a bug and should be fixed in the 2.7 tree. Perhaps the documentation in older versions should be updated 

mmap.mmap should always set the FD_CLOEXEC flag on the descriptor that it gets from dup(). I don't see why we wouldn't do this, because if we are calling exec(), we are blowing away all python state, and only the mmap object should be using this file descriptor (since its existence is hidden from the user).

Users should not be depending on the old behavior, because the interface which is presented to them gives them no indication that this fd even exists. Users are probably expecting the proposed behavior, because anyone who has worked with unix file descriptors in C would not expect:

fd = os.open(...)
mmap = mmap.mmap(fd, size)
# do some stuff
os.close(fd)

To result in an extra inheritable fd still being there. Nothing in the documentation indicates that this would happen, either.
msg201726 - (view) Author: Robert Merrill (rfm) Date: 2013-10-30 10:34
Sorry, I correct my earlier statement: even if the fd you pass to mmap.mmap() is set to FD_CLOEXEC, the dup'd fd /will not be/

So this is a REALLY bad bug because users cannot workaround it except by just not using mmap
msg201755 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-10-30 17:47
I agree this should be fixed.
Robert, want to submit a patch?
msg201854 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-10-31 22:24
> Robert, want to submit a patch?

If someone wants to work on a patch, here is the code of Python 3.4 to duplicate a file descriptor and make it non-inheritable.
  http://hg.python.org/cpython/file/e97d9926da83/Python/fileutils.c#l978
and
  http://hg.python.org/cpython/file/e97d9926da83/Python/fileutils.c#l618

The code is complex because it has optimized versions for each operating system. Only supporting Windows and fcntl() is enough for Python 2.
msg265050 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-05-07 08:13
Closing because Victor has proposed a patch in Issue 26769.
History
Date User Action Args
2022-04-11 14:57:52adminsetgithub: 63643
2016-05-07 08:13:57martin.pantersetstatus: open -> closed

nosy: + martin.panter
messages: + msg265050

superseder: Python 2.7: make private file descriptors non inheritable
resolution: duplicate
2016-04-15 09:16:22berker.peksagsetassignee: docs@python ->
components: - Documentation
versions: - Python 3.3
2013-10-31 22:24:11vstinnersetmessages: + msg201854
2013-10-30 23:11:54Arfreversetnosy: + Arfrever
2013-10-30 17:47:07neologixsetmessages: + msg201755
2013-10-30 10:34:27rfmsetmessages: + msg201726
2013-10-30 09:45:36rfmsetmessages: + msg201722
components: + Library (Lib)
versions: - Python 3.4
2013-10-30 09:35:24vstinnersetnosy: + neologix
2013-10-30 09:33:43vstinnersetmessages: + msg201720
versions: + Python 3.3, Python 3.4, - Python 3.2
2013-10-30 09:23:44christian.heimessetassignee: docs@python
components: + Documentation, - Library (Lib)
versions: + Python 3.2
nosy: + christian.heimes, docs@python

messages: + msg201718
stage: needs patch
2013-10-30 09:12:06pitrousetnosy: + vstinner
2013-10-30 05:25:24rfmsetmessages: + msg201710
2013-10-30 05:19:09rfmcreate