classification
Title: os.O_CLOEXEC not available on OS X
Type: Stage: resolved
Components: macOS Versions: Python 3.5
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Gustavo Goretkin, ned.deily, ronaldoussoren, vstinner
Priority: normal Keywords:

Created on 2016-02-11 22:11 by Gustavo Goretkin, last changed 2016-02-12 10:25 by vstinner. This issue is now closed.

Messages (9)
msg260135 - (view) Author: Gustavo Goretkin (Gustavo Goretkin) Date: 2016-02-11 22:11
I am on OS X 10.9.5

Python 3.5.1 (v3.5.1:37a07cee5969, Dec  5 2015, 21:12:44) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.O_CLOEXEC
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'os' has no attribute 'O_CLOEXEC'

I checked on my system
$ man 2 open | grep CLOEXEC
           O_CLOEXEC       mark as close-on-exec
     The O_CLOEXEC flag causes the file descriptor to be marked as close-on-
     exec, setting the FD_CLOEXEC flag.  The state of the file descriptor

I first noticed this on an anaconda distribution of python, but it looks like it is also present on the 3.5 .dmg file on https://www.python.org/downloads/
msg260137 - (view) Author: Gustavo Goretkin (Gustavo Goretkin) Date: 2016-02-11 22:32
It looks like cpython will check to see if O_CLOEXEC is #defined. 

In my system's /usr/include/sys/fcntl.h

#if __DARWIN_C_LEVEL >= 200809L
#define O_CLOEXEC       0x1000000       /* implicitly set FD_CLOEXEC */
#endif
msg260139 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-02-11 22:43
FYI Python creates non-inheritable file descriptors by default since Python 3.4: PEP 446.

See also the PEP 433 which has more information on OS support, especially:
https://www.python.org/dev/peps/pep-0433/#atomic-flags
msg260140 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-02-11 22:44
What is __DARWIN_C_LEVEL? Are we supposed to define it directly? Or is it defined from other #define in features.h?
msg260141 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2016-02-11 22:50
The lack of o_cloexec could be caused by the SDK used to build the binary installer. AFAIK _DARWIN_C_LEVEL is set automatically based on the deployment target. 

Ronald

Sent from my iPad

> On 11 feb. 2016, at 23:44, STINNER Victor <report@bugs.python.org> wrote:
> 
> 
> STINNER Victor added the comment:
> 
> What is __DARWIN_C_LEVEL? Are we supposed to define it directly? Or is it defined from other #define in features.h?
> 
> ----------
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue26343>
> _______________________________________
msg260151 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-02-12 00:47
Ronald is correct.  Apple added O_CLOEXEC to OS X in 10.7.  For compatibility reasons, the python.org 64-bit installers for OS X currently are built to be compatible with OS X 10.6 on up and do not do runtime checks for OS features only available on newer releases.  If you need O_CLOEXEC support, you could use a Python from another distributor, such as Homebrew or MacPorts, or you could build it from source yourself.


https://developer.apple.com/library/mac/releasenotes/General/MacOSXLionAPIDiffs/Kernel.html
msg260163 - (view) Author: Gustavo Goretkin (Gustavo Goretkin) Date: 2016-02-12 04:48
Can the documentation be updated accordingly? Currently the documentation (roughly) says that this is available.

If not separately noted, all functions that claim “Availability: Unix” are supported on Mac OS X, which builds on a Unix core.

....
os.O_EXLOCK
os.O_CLOEXEC

    These constants are only available on Unix.
msg260164 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-02-12 05:01
This is only one example of many cases of calls and options that are exposed by the os module but are not available on specific platforms or platform releases.  It would be a huge documentation effort to try to identify and keep them all up-to-date.  The fourth bullet at the beginning of the "os" module doc page is intended to cover this:

"An “Availability: Unix” note means that this function is commonly found on Unix systems. It does not make any claims about its existence on a specific operating system."

https://docs.python.org/3/library/os.html#module-os
msg260169 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-02-12 10:25
> FYI Python creates non-inheritable file descriptors by default since Python 3.4: PEP 446.

You didn't reply why you need O_CLOEXEC?
History
Date User Action Args
2016-02-12 10:25:22vstinnersetmessages: + msg260169
2016-02-12 05:01:54ned.deilysetmessages: + msg260164
2016-02-12 04:48:19Gustavo Goretkinsetmessages: + msg260163
2016-02-12 00:47:52ned.deilysetstatus: open -> closed
resolution: not a bug
messages: + msg260151

stage: resolved
2016-02-11 22:50:54ronaldoussorensetmessages: + msg260141
2016-02-11 22:44:24vstinnersetmessages: + msg260140
2016-02-11 22:43:52vstinnersetnosy: + vstinner
messages: + msg260139
2016-02-11 22:32:49Gustavo Goretkinsetmessages: + msg260137
2016-02-11 22:11:23Gustavo Goretkincreate