diff -r f87d85f7596c -r 9bdfa1a3ea8c Doc/library/asyncore.rst --- a/Doc/library/asyncore.rst Wed Jan 16 15:19:16 2013 +0100 +++ b/Doc/library/asyncore.rst Sat Jan 26 01:09:28 2013 +0100 @@ -184,15 +184,20 @@ any that have been added to the map duri Most of these are nearly identical to their socket partners. - .. method:: create_socket(family=socket.AF_INET, type=socket.SOCK_STREAM) + .. method:: create_socket(family=socket.AF_INET, type=socket.SOCK_STREAM, cloexec=None) This is identical to the creation of a normal socket, and will use the same options for creation. Refer to the :mod:`socket` documentation for information on creating sockets. + If *cloexec* is ``True``, set the :ref:`close-on-exec flag `. + .. versionchanged:: 3.3 *family* and *type* arguments can be omitted. + .. versionchanged:: 3.4 + *cloexec* parameter was added. + .. method:: connect(address) diff -r f87d85f7596c -r 9bdfa1a3ea8c Doc/library/functions.rst --- a/Doc/library/functions.rst Wed Jan 16 15:19:16 2013 +0100 +++ b/Doc/library/functions.rst Sat Jan 26 01:09:28 2013 +0100 @@ -823,7 +823,7 @@ are always available. They are listed h .. index:: single: file object; open() built-in function -.. function:: open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None) +.. function:: open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None, cloexec=None) Open *file* and return a corresponding :term:`file object`. If the file cannot be opened, an :exc:`OSError` is raised. @@ -942,6 +942,8 @@ are always available. They are listed h :mod:`os.open` as *opener* results in functionality similar to passing ``None``). + If *cloexec* is ``True``, set the :ref:`close-on-exec flag `. + The following example uses the :ref:`dir_fd ` parameter of the :func:`os.open` function to open a file relative to a given directory:: @@ -959,6 +961,9 @@ are always available. They are listed h The *opener* parameter was added. The ``'x'`` mode was added. + .. versionchanged:: 3.4 + The *cloexec* parameter was added. + The type of :term:`file object` returned by the :func:`open` function depends on the mode. When :func:`open` is used to open a file in a text mode (``'w'``, ``'r'``, ``'wt'``, ``'rt'``, etc.), it returns a subclass of diff -r f87d85f7596c -r 9bdfa1a3ea8c Doc/library/io.rst --- a/Doc/library/io.rst Wed Jan 16 15:19:16 2013 +0100 +++ b/Doc/library/io.rst Sat Jan 26 01:09:28 2013 +0100 @@ -110,7 +110,7 @@ High-level Module Interface :func:`os.stat`) if possible. -.. function:: open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True) +.. function:: open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, cloexec=None) This is an alias for the builtin :func:`open` function. @@ -487,7 +487,7 @@ I/O Base Classes Raw File I/O ^^^^^^^^^^^^ -.. class:: FileIO(name, mode='r', closefd=True, opener=None) +.. class:: FileIO(name, mode='r', closefd=True, opener=None, cloexec=None) :class:`FileIO` represents an OS-level file containing bytes data. It implements the :class:`RawIOBase` interface (and therefore the @@ -517,6 +517,8 @@ Raw File I/O :mod:`os.open` as *opener* results in functionality similar to passing ``None``). + If *cloexec* is ``True``, set the :ref:`close-on-exec flag `. + See the :func:`open` built-in function for examples on using the *opener* parameter. @@ -524,6 +526,9 @@ Raw File I/O The *opener* parameter was added. The ``'x'`` mode was added. + .. versionchanged:: 3.4 + The *cloexec* parameter was added. + In addition to the attributes and methods from :class:`IOBase` and :class:`RawIOBase`, :class:`FileIO` provides the following data attributes: diff -r f87d85f7596c -r 9bdfa1a3ea8c Doc/library/os.rst --- a/Doc/library/os.rst Wed Jan 16 15:19:16 2013 +0100 +++ b/Doc/library/os.rst Sat Jan 26 01:09:28 2013 +0100 @@ -683,19 +683,27 @@ as internal buffering of data. if it is connected to a terminal; else return :const:`None`. -.. function:: dup(fd) - - Return a duplicate of file descriptor *fd*. +.. function:: dup(fd, cloexec=None) + + Return a duplicate of file descriptor *fd*. If *cloexec* is ``True``, set + the :ref:`close-on-exec flag `. Availability: Unix, Windows. - -.. function:: dup2(fd, fd2) + .. versionchanged:: 3.4 + *cloexec* parameter was added. + + +.. function:: dup2(fd, fd2, cloexec=None) Duplicate file descriptor *fd* to *fd2*, closing the latter first if necessary. + If *cloexec* is ``True``, set the :ref:`close-on-exec flag ` on *fd2*. Availability: Unix, Windows. + .. versionchanged:: 3.4 + *cloexec* parameter was added. + .. function:: fchmod(fd, mode) @@ -843,7 +851,7 @@ as internal buffering of data. :data:`os.SEEK_HOLE` or :data:`os.SEEK_DATA`. -.. function:: open(file, flags, mode=0o777, *, dir_fd=None) +.. function:: open(file, flags, mode=0o777, *, dir_fd=None, cloexec=None) Open the file *file* and set various flags according to *flags* and possibly its mode according to *mode*. When computing *mode*, the current umask value @@ -857,8 +865,13 @@ as internal buffering of data. This function can support :ref:`paths relative to directory descriptors `. + If *cloexec* is ``True``, set the :ref:`close-on-exec flag `. + Availability: Unix, Windows. + .. versionchanged:: 3.4 + *cloexec* parameter was added. + .. note:: This function is intended for low-level I/O. For normal usage, use the @@ -870,24 +883,34 @@ as internal buffering of data. The *dir_fd* argument. -.. function:: openpty() +.. function:: openpty(cloexec=None) .. index:: module: pty - Open a new pseudo-terminal pair. Return a pair of file descriptors ``(master, - slave)`` for the pty and the tty, respectively. For a (slightly) more portable - approach, use the :mod:`pty` module. + Open a new pseudo-terminal pair. If *cloexec* is ``True``, set + the :ref:`close-on-exec flag `. Return a pair of file descriptors + ``(master, slave)`` for the pty and the tty, respectively. For a (slightly) + more portable approach, use the :mod:`pty` module. Availability: some flavors of Unix. - -.. function:: pipe() - - Create a pipe. Return a pair of file descriptors ``(r, w)`` usable for reading + .. versionchanged:: 3.4 + *cloexec* parameter was added. + + +.. function:: pipe(cloexec=None) + + Create a pipe. If *cloexec* is ``True``, set the :ref:`close-on-exec flag + `. Return a pair of file descriptors ``(r, w)`` usable for reading and writing, respectively. + Setting close-on-exec flag is atomic on Windows and Linux 2.6.27 or newer. + Availability: Unix, Windows. + .. versionchanged:: 3.4 + *cloexec* parameter was added. + .. function:: pipe2(flags) @@ -1192,6 +1215,51 @@ Querying the size of a terminal Height of the terminal window in characters. +.. _cloexec: + +Close-on-exec flag +~~~~~~~~~~~~~~~~~~ + +A file descriptor has a close-on-exec flag which indicates if the file +descriptor will be inherited or not. + +On UNIX, the file descriptor will be closed on the execution of child processes +if the close-on-exec flag is set, the file descriptor is inherited by child +processes if the flag is cleared. + +On Windows, the file descriptor is not inherited if the close-on-exec flag is +set, the file descriptor is inherited by child processes if the flag is cleared +and if :c:func:`CreateProcess` is called with the *bInheritHandles* parameter +set to ``TRUE`` (when :class:`subprocess.Popen` is created with +``close_fds=False`` for example). + +Example of functions having the *cloexec* parameter: :func:`open`, +:func:`os.pipe`, :func:`socket.socket`. The default value of the *cloexec* +parameter is :func:`sys.getdefaultcloexec`, it is ``False`` at startup. It +can be set to ``True`` using :func:`sys.setdefaultcloexec`, by setting the +:envvar:`PYTHONCLOEXEC` environment variable, and using :option:`-e` command +line option. + + +.. function:: get_cloexec(fd) + + Get close-on-exe flag of the specified file descriptor. Return a :class:`bool`. + + Availability: Windows, Linux, FreeBSD, NetBSD, OpenBSD, Mac OS X, Solaris. + + .. versionadded:: 3.4 + +.. function:: set_cloexec(fd, cloexec=True) + + Set or clear close-on-exe flag on the specified file descriptor. + + Availability: Windows, Linux, FreeBSD, NetBSD, OpenBSD, Mac OS X, Solaris. + + .. versionadded:: 3.4 + + + + .. _os-file-dir: Files and Directories diff -r f87d85f7596c -r 9bdfa1a3ea8c Doc/library/socket.rst --- a/Doc/library/socket.rst Wed Jan 16 15:19:16 2013 +0100 +++ b/Doc/library/socket.rst Sat Jan 26 01:09:28 2013 +0100 @@ -445,7 +445,7 @@ The module :mod:`socket` exports the fol ``'udp'``, otherwise any protocol will match. -.. function:: socket([family[, type[, proto]]]) +.. function:: socket(family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None, cloexec=None) Create a new socket using the given address family, socket type and protocol number. The address family should be :const:`AF_INET` (the default), @@ -455,25 +455,36 @@ The module :mod:`socket` exports the fol constants. The protocol number is usually zero and may be omitted in that case or :const:`CAN_RAW` in case the address family is :const:`AF_CAN`. + If *cloexec* is ``True``, set the :ref:`close-on-exec flag `. + .. versionchanged:: 3.3 The AF_CAN family was added. The AF_RDS family was added. + .. versionchanged:: 3.4 + *cloexec* parameter was added. -.. function:: socketpair([family[, type[, proto]]]) + +.. function:: socketpair(family=AF_INET, type=SOCK_STREAM, proto=0, cloexec=None) Build a pair of connected socket objects using the given address family, socket type, and protocol number. Address family, socket type, and protocol number are as for the :func:`socket` function above. The default family is :const:`AF_UNIX` if defined on the platform; otherwise, the default is :const:`AF_INET`. + + If *cloexec* is ``True``, set the :ref:`close-on-exec flag `. + Availability: Unix. .. versionchanged:: 3.2 The returned socket objects now support the whole socket API, rather than a subset. + .. versionchanged:: 3.4 + *cloexec* parameter was added. -.. function:: fromfd(fd, family, type[, proto]) + +.. function:: fromfd(fd, family, type, proto=0) Duplicate the file descriptor *fd* (an integer as returned by a file object's :meth:`fileno` method) and build a socket object from the result. Address @@ -705,13 +716,18 @@ Socket objects have the following method correspond to Unix system calls applicable to sockets. -.. method:: socket.accept() +.. method:: socket.accept(cloexec=None) Accept a connection. The socket must be bound to an address and listening for connections. The return value is a pair ``(conn, address)`` where *conn* is a *new* socket object usable to send and receive data on the connection, and *address* is the address bound to the socket on the other end of the connection. + If *cloexec* is ``True``, set the :ref:`close-on-exec flag `. + + .. versionchanged:: 3.4 + *cloexec* parameter was added. + .. method:: socket.bind(address) diff -r f87d85f7596c -r 9bdfa1a3ea8c Doc/library/sys.rst --- a/Doc/library/sys.rst Wed Jan 16 15:19:16 2013 +0100 +++ b/Doc/library/sys.rst Sat Jan 26 01:09:28 2013 +0100 @@ -401,6 +401,16 @@ always available. Use :func:`getswitchinterval` instead. +.. function:: getdefaultcloexec() + + Return the default value of the *cloexec* parameter: + see :ref:`close-on-exec flag `. + + .. seealso:: :func:`sys.setdefaultcloexec`. + + .. versionadded:: 3.4 + + .. function:: getdefaultencoding() Return the name of the current default string encoding used by the Unicode @@ -903,6 +913,16 @@ always available. :func:`setswitchinterval` instead. +.. function:: setdefaultcloexec() + + Set the default value of the *cloexec* parameter to ``True``: + see the :ref:`close-on-exec flag `. + + .. seealso:: :func:`sys.getdefaultcloexec`. + + .. versionadded:: 3.4 + + .. function:: setdlopenflags(n) Set the flags used by the interpreter for :c:func:`dlopen` calls, such as when diff -r f87d85f7596c -r 9bdfa1a3ea8c Doc/using/cmdline.rst --- a/Doc/using/cmdline.rst Wed Jan 16 15:19:16 2013 +0100 +++ b/Doc/using/cmdline.rst Sat Jan 26 01:09:28 2013 +0100 @@ -191,6 +191,14 @@ Miscellaneous options options). See also :envvar:`PYTHONDEBUG`. +.. cmdoption:: -e + + Set the default value of the *cloexec* parameter to ``True``: see the + :ref:`close-on-exec flag `. + + .. versionadded:: 3.4 + + .. cmdoption:: -E Ignore all :envvar:`PYTHON*` environment variables, e.g. @@ -559,6 +567,17 @@ conflict. Python traceback. This is equivalent to :option:`-X` ``faulthandler`` option. + .. versionadded:: 3.3 + +.. envvar:: PYTHONCLOEXEC + + If this is set, set the default value of the *cloexec* parameter to + ``True``: see the :ref:`close-on-exec flag `. This is equivalent to + :option:`-e` option. + + .. versionadded:: 3.4 + + Debug-mode variables ~~~~~~~~~~~~~~~~~~~~ diff -r f87d85f7596c -r 9bdfa1a3ea8c Include/fileutils.h --- a/Include/fileutils.h Wed Jan 16 15:19:16 2013 +0100 +++ b/Include/fileutils.h Sat Jan 26 01:09:28 2013 +0100 @@ -5,6 +5,8 @@ extern "C" { #endif +PyAPI_DATA(int) Py_DefaultCloexec; + PyAPI_FUNC(PyObject *) _Py_device_encoding(int); PyAPI_FUNC(wchar_t *) _Py_char2wchar( @@ -27,6 +29,14 @@ PyAPI_FUNC(int) _Py_stat( struct stat *statbuf); #endif +PyAPI_FUNC(int) _Py_open( + const char *pathname, + int flags); + +PyAPI_FUNC(int) _Py_open_cloexec( + const char *pathname, + int flags); + PyAPI_FUNC(FILE *) _Py_wfopen( const wchar_t *path, const wchar_t *mode); @@ -53,6 +63,16 @@ PyAPI_FUNC(wchar_t*) _Py_wgetcwd( wchar_t *buf, size_t size); +PyAPI_FUNC(int) _Py_get_cloexec(int fd); + +PyAPI_FUNC(int) _Py_set_cloexec(int fd, int cloexec, int *atomic_flags_works); + +PyAPI_FUNC(void) _Py_try_set_cloexec(int fd, int cloexec); + +PyAPI_FUNC(void) _Py_try_set_default_cloexec(int fd); + +PyAPI_FUNC(int) _Py_cloexec_converter(PyObject* arg, void* addr); + #ifdef __cplusplus } #endif diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/_pyio.py --- a/Lib/_pyio.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/_pyio.py Sat Jan 26 01:09:28 2013 +0100 @@ -32,7 +32,7 @@ BlockingIOError = BlockingIOError def open(file, mode="r", buffering=-1, encoding=None, errors=None, - newline=None, closefd=True, opener=None): + newline=None, closefd=True, opener=None, cloexec=None): r"""Open file and return a stream. Raise OSError upon failure. @@ -135,6 +135,8 @@ def open(file, mode="r", buffering=-1, e descriptor (passing os.open as *opener* results in functionality similar to passing None). + If cloexec is True, the close-on-exec flag is set. + open() returns a file object whose type depends on the mode, and through which the standard file operations such as reading and writing are performed. When open() is used to open a file in a text mode ('w', @@ -191,7 +193,7 @@ def open(file, mode="r", buffering=-1, e (writing and "w" or "") + (appending and "a" or "") + (updating and "+" or ""), - closefd, opener=opener) + closefd, opener=opener, cloexec=cloexec) line_buffering = False if buffering == 1 or buffering < 0 and raw.isatty(): buffering = -1 diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/asyncore.py --- a/Lib/asyncore.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/asyncore.py Sat Jan 26 01:09:28 2013 +0100 @@ -284,9 +284,9 @@ class dispatcher: del map[fd] self._fileno = None - def create_socket(self, family=socket.AF_INET, type=socket.SOCK_STREAM): + def create_socket(self, family=socket.AF_INET, type=socket.SOCK_STREAM, cloexec=True): self.family_and_type = family, type - sock = socket.socket(family, type) + sock = socket.socket(family, type, cloexec=cloexec) sock.setblocking(0) self.set_socket(sock) diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/cgi.py --- a/Lib/cgi.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/cgi.py Sat Jan 26 01:09:28 2013 +0100 @@ -79,7 +79,7 @@ def initlog(*allargs): global log, logfile, logfp if logfile and not logfp: try: - logfp = open(logfile, "a") + logfp = open(logfile, "a", cloexec=True) except OSError: pass if not logfp: diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/getpass.py --- a/Lib/getpass.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/getpass.py Sat Jan 26 01:09:28 2013 +0100 @@ -42,8 +42,8 @@ def unix_getpass(prompt='Password: ', st tty = None try: # Always try reading and writing directly on the tty first. - fd = os.open('/dev/tty', os.O_RDWR|os.O_NOCTTY) - tty = os.fdopen(fd, 'w+', 1) + fd = os.open('/dev/tty', os.O_RDWR|os.O_NOCTTY, cloexec=True) + tty = os.fdopen(fd, 'w+', 1, cloexec=False) input = tty if not stream: stream = tty diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/http/server.py --- a/Lib/http/server.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/http/server.py Sat Jan 26 01:09:28 2013 +0100 @@ -710,7 +710,7 @@ class SimpleHTTPRequestHandler(BaseHTTPR return self.list_directory(path) ctype = self.guess_type(path) try: - f = open(path, 'rb') + f = open(path, 'rb', cloexec=True) except OSError: self.send_error(404, "File not found") return None @@ -1125,8 +1125,8 @@ class CGIHTTPRequestHandler(SimpleHTTPRe os.setuid(nobody) except OSError: pass - os.dup2(self.rfile.fileno(), 0) - os.dup2(self.wfile.fileno(), 1) + os.dup2(self.rfile.fileno(), 0, cloexec=False) + os.dup2(self.wfile.fileno(), 1, cloexec=False) os.execve(scriptfile, args, env) except: self.server.handle_error(self.request, self.client_address) diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/importlib/_bootstrap.py --- a/Lib/importlib/_bootstrap.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/importlib/_bootstrap.py Sat Jan 26 01:09:28 2013 +0100 @@ -125,7 +125,9 @@ def _write_atomic(path, data, mode=0o666 # id() is used to generate a pseudo-random filename. path_tmp = '{}.{}'.format(path, id(path)) fd = _os.open(path_tmp, - _os.O_EXCL | _os.O_CREAT | _os.O_WRONLY, mode & 0o666) + _os.O_EXCL | _os.O_CREAT | _os.O_WRONLY, + mode & 0o666, + cloexec=True) try: # We first write data to a temporary file, and then use os.replace() to # perform an atomic rename. diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/logging/__init__.py --- a/Lib/logging/__init__.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/logging/__init__.py Sat Jan 26 01:09:28 2013 +0100 @@ -994,7 +994,8 @@ class FileHandler(StreamHandler): Open the current base file with the (original) mode and encoding. Return the resulting stream. """ - return open(self.baseFilename, self.mode, encoding=self.encoding) + return open(self.baseFilename, self.mode, + encoding=self.encoding, cloexec=True) def emit(self, record): """ diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/multiprocessing/forking.py --- a/Lib/multiprocessing/forking.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/multiprocessing/forking.py Sat Jan 26 01:09:28 2013 +0100 @@ -211,13 +211,13 @@ else: prep_data = get_preparation_data(process_obj._name) # create pipe for communication with child - rfd, wfd = os.pipe() + rfd, wfd = os.pipe(cloexec=False) # get handle for read end of the pipe and make it inheritable rhandle = duplicate(msvcrt.get_osfhandle(rfd), inheritable=True) os.close(rfd) - with open(wfd, 'wb', closefd=True) as to_child: + with open(wfd, 'wb', closefd=True, cloexec=False) as to_child: # start process try: hp, ht, pid, tid = _winapi.CreateProcess( @@ -337,7 +337,7 @@ else: handle = int(sys.argv[-1]) fd = msvcrt.open_osfhandle(handle, os.O_RDONLY) - from_parent = os.fdopen(fd, 'rb') + from_parent = os.fdopen(fd, 'rb', cloexec=True) process.current_process()._inheriting = True preparation_data = load(from_parent) diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/multiprocessing/process.py --- a/Lib/multiprocessing/process.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/multiprocessing/process.py Sat Jan 26 01:09:28 2013 +0100 @@ -241,7 +241,7 @@ class Process(object): if sys.stdin is not None: try: sys.stdin.close() - sys.stdin = open(os.devnull) + sys.stdin = open(os.devnull, cloexec=False) except (OSError, ValueError): pass old_process = _current_process diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/os.py --- a/Lib/os.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/os.py Sat Jan 26 01:09:28 2013 +0100 @@ -439,7 +439,7 @@ if {open, stat} <= supports_dir_fd and { # Note: To guard against symlink races, we use the standard # lstat()/open()/fstat() trick. orig_st = stat(top, follow_symlinks=False, dir_fd=dir_fd) - topfd = open(top, O_RDONLY, dir_fd=dir_fd) + topfd = open(top, O_RDONLY, dir_fd=dir_fd, cloexec=True) try: if (follow_symlinks or (st.S_ISDIR(orig_st.st_mode) and path.samestat(orig_st, stat(topfd)))): @@ -479,7 +479,7 @@ if {open, stat} <= supports_dir_fd and { for name in dirs: try: orig_st = stat(name, dir_fd=topfd, follow_symlinks=follow_symlinks) - dirfd = open(name, O_RDONLY, dir_fd=topfd) + dirfd = open(name, O_RDONLY, dir_fd=topfd, cloexec=True) except OSError as err: if onerror is not None: onerror(err) diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/pty.py --- a/Lib/pty.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/pty.py Sat Jan 26 01:09:28 2013 +0100 @@ -63,7 +63,7 @@ def _open_terminal(): for y in '0123456789abcdef': pty_name = '/dev/pty' + x + y try: - fd = os.open(pty_name, os.O_RDWR) + fd = os.open(pty_name, os.O_RDWR, cloexec=True) except OSError: continue return (fd, '/dev/tty' + x + y) @@ -75,7 +75,7 @@ def slave_open(tty_name): opened filedescriptor. Deprecated, use openpty() instead.""" - result = os.open(tty_name, os.O_RDWR) + result = os.open(tty_name, os.O_RDWR, cloexec=True) try: from fcntl import ioctl, I_PUSH except ImportError: @@ -112,14 +112,14 @@ def fork(): os.close(master_fd) # Slave becomes stdin/stdout/stderr of child. - os.dup2(slave_fd, STDIN_FILENO) - os.dup2(slave_fd, STDOUT_FILENO) - os.dup2(slave_fd, STDERR_FILENO) + os.dup2(slave_fd, STDIN_FILENO, cloexec=False) + os.dup2(slave_fd, STDOUT_FILENO, cloexec=False) + os.dup2(slave_fd, STDERR_FILENO, cloexec=False) if (slave_fd > STDERR_FILENO): os.close (slave_fd) # Explicitly open the tty to make it become a controlling tty. - tmp_fd = os.open(os.ttyname(STDOUT_FILENO), os.O_RDWR) + tmp_fd = os.open(os.ttyname(STDOUT_FILENO), os.O_RDWR, cloexec=True) os.close(tmp_fd) else: os.close(slave_fd) diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/pydoc.py --- a/Lib/pydoc.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/pydoc.py Sat Jan 26 01:09:28 2013 +0100 @@ -265,7 +265,7 @@ class ErrorDuringImport(Exception): def importfile(path): """Import a Python source file or compiled file given its path.""" magic = imp.get_magic() - with open(path, 'rb') as file: + with open(path, 'rb', cloexec=True) as file: if file.read(len(magic)) == magic: kind = imp.PY_COMPILED else: @@ -1426,7 +1426,7 @@ def tempfilepager(text, cmd): """Page through text by invoking a program on a temporary file.""" import tempfile filename = tempfile.mktemp() - file = open(filename, 'w') + file = open(filename, 'w', cloexec=True) file.write(text) file.close() try: @@ -1580,7 +1580,7 @@ def writedoc(thing, forceload=0): try: object, name = resolve(thing, forceload) page = html.page(describe(object), html.document(object, name)) - file = open(name + '.html', 'w', encoding='utf-8') + file = open(name + '.html', 'w', encoding='utf-8', cloexec=True) file.write(page) file.close() print('wrote', name + '.html') @@ -2471,7 +2471,7 @@ def _url_handler(url, content_type="text if content_type == 'text/css': path_here = os.path.dirname(os.path.realpath(__file__)) css_path = os.path.join(path_here, url) - with open(css_path) as fp: + with open(css_path, cloexec=True) as fp: return ''.join(fp.readlines()) elif content_type == 'text/html': return get_html_page(url) diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/shutil.py --- a/Lib/shutil.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/shutil.py Sat Jan 26 01:09:28 2013 +0100 @@ -104,8 +104,8 @@ def copyfile(src, dst, *, follow_symlink if not follow_symlinks and os.path.islink(src): os.symlink(os.readlink(src), dst) else: - with open(src, 'rb') as fsrc: - with open(dst, 'wb') as fdst: + with open(src, 'rb', cloexec=True) as fsrc: + with open(dst, 'wb', cloexec=True) as fdst: copyfileobj(fsrc, fdst) return dst @@ -386,7 +386,7 @@ def _rmtree_safe_fd(topfd, path, onerror mode = 0 if stat.S_ISDIR(mode): try: - dirfd = os.open(name, os.O_RDONLY, dir_fd=topfd) + dirfd = os.open(name, os.O_RDONLY, dir_fd=topfd, cloexec=True) except OSError: onerror(os.open, fullname, sys.exc_info()) else: @@ -448,7 +448,7 @@ def rmtree(path, ignore_errors=False, on onerror(os.lstat, path, sys.exc_info()) return try: - fd = os.open(path, os.O_RDONLY) + fd = os.open(path, os.O_RDONLY, cloexec=True) except Exception: onerror(os.lstat, path, sys.exc_info()) return @@ -875,7 +875,7 @@ def _unpack_zipfile(filename, extract_di if not name.endswith('/'): # file data = zip.read(info.filename) - f = open(target, 'wb') + f = open(target, 'wb', cloexec=True) try: f.write(data) finally: diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/socket.py --- a/Lib/socket.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/socket.py Sat Jan 26 01:09:28 2013 +0100 @@ -90,8 +90,8 @@ class socket(_socket.socket): __slots__ = ["__weakref__", "_io_refs", "_closed"] - def __init__(self, family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None): - _socket.socket.__init__(self, family, type, proto, fileno) + def __init__(self, family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None, cloexec=None): + _socket.socket.__init__(self, family, type, proto, fileno, cloexec) self._io_refs = 0 self._closed = False @@ -125,14 +125,14 @@ class socket(_socket.socket): sock.settimeout(self.gettimeout()) return sock - def accept(self): + def accept(self, cloexec=None): """accept() -> (socket object, address info) Wait for an incoming connection. Return a new socket representing the connection, and the address of the client. For IP sockets, the address info is a pair (hostaddr, port). """ - fd, addr = self._accept() + fd, addr = self._accept(cloexec=cloexec) sock = socket(self.family, self.type, self.proto, fileno=fd) # Issue #7995: if no default timeout is set and the listening # socket had a (non-zero) timeout, force the new socket in blocking @@ -230,7 +230,7 @@ if hasattr(_socket.socket, "share"): if hasattr(_socket, "socketpair"): - def socketpair(family=None, type=SOCK_STREAM, proto=0): + def socketpair(family=None, type=SOCK_STREAM, proto=0, cloexec=None): """socketpair([family[, type[, proto]]]) -> (socket object, socket object) Create a pair of socket objects from the sockets returned by the platform @@ -243,9 +243,9 @@ if hasattr(_socket, "socketpair"): family = AF_UNIX except NameError: family = AF_INET - a, b = _socket.socketpair(family, type, proto) - a = socket(family, type, proto, a.detach()) - b = socket(family, type, proto, b.detach()) + a, b = _socket.socketpair(family, type, proto, cloexec=cloexec) + a = socket(family, type, proto, a.detach(), cloexec=False) + b = socket(family, type, proto, b.detach(), cloexec=False) return a, b diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/subprocess.py --- a/Lib/subprocess.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/subprocess.py Sat Jan 26 01:09:28 2013 +0100 @@ -400,7 +400,6 @@ else: import select _has_poll = hasattr(select, 'poll') import _posixsubprocess - _create_pipe = _posixsubprocess.cloexec_pipe # When select or poll has indicated that the file is writable, # we can write up to _PIPE_BUF bytes without risk of blocking. @@ -794,15 +793,15 @@ class Popen(object): errread = msvcrt.open_osfhandle(errread.Detach(), 0) if p2cwrite != -1: - self.stdin = io.open(p2cwrite, 'wb', bufsize) + self.stdin = io.open(p2cwrite, 'wb', bufsize, cloexec=False) if universal_newlines: self.stdin = io.TextIOWrapper(self.stdin, write_through=True) if c2pread != -1: - self.stdout = io.open(c2pread, 'rb', bufsize) + self.stdout = io.open(c2pread, 'rb', bufsize, cloexec=False) if universal_newlines: self.stdout = io.TextIOWrapper(self.stdout) if errread != -1: - self.stderr = io.open(errread, 'rb', bufsize) + self.stderr = io.open(errread, 'rb', bufsize, cloexec=False) if universal_newlines: self.stderr = io.TextIOWrapper(self.stderr) @@ -871,7 +870,7 @@ class Popen(object): def _get_devnull(self): if not hasattr(self, '_devnull'): - self._devnull = os.open(os.devnull, os.O_RDWR) + self._devnull = os.open(os.devnull, os.O_RDWR, cloexec=False) return self._devnull def communicate(self, input=None, timeout=None): @@ -1230,7 +1229,7 @@ class Popen(object): if stdin is None: pass elif stdin == PIPE: - p2cread, p2cwrite = _create_pipe() + p2cread, p2cwrite = os.pipe(cloexec=True) elif stdin == DEVNULL: p2cread = self._get_devnull() elif isinstance(stdin, int): @@ -1242,7 +1241,7 @@ class Popen(object): if stdout is None: pass elif stdout == PIPE: - c2pread, c2pwrite = _create_pipe() + c2pread, c2pwrite = os.pipe(cloexec=True) elif stdout == DEVNULL: c2pwrite = self._get_devnull() elif isinstance(stdout, int): @@ -1254,7 +1253,7 @@ class Popen(object): if stderr is None: pass elif stderr == PIPE: - errread, errwrite = _create_pipe() + errread, errwrite = os.pipe(cloexec=True) elif stderr == STDOUT: errwrite = c2pwrite elif stderr == DEVNULL: @@ -1306,7 +1305,7 @@ class Popen(object): # For transferring possible exec failure from child to parent. # Data format: "exception name:hex errno:description" # Pickle is not used; it is complex and involves memory allocation. - errpipe_read, errpipe_write = _create_pipe() + errpipe_read, errpipe_write = os.pipe(cloexec=True) try: try: # We must avoid complex work that could involve diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/tempfile.py --- a/Lib/tempfile.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/tempfile.py Sat Jan 26 01:09:28 2013 +0100 @@ -34,33 +34,12 @@ import os as _os from random import Random as _Random try: - import fcntl as _fcntl -except ImportError: - def _set_cloexec(fd): - pass -else: - def _set_cloexec(fd): - try: - flags = _fcntl.fcntl(fd, _fcntl.F_GETFD, 0) - except OSError: - pass - else: - # flags read successfully, modify - flags |= _fcntl.FD_CLOEXEC - _fcntl.fcntl(fd, _fcntl.F_SETFD, flags) - - -try: import _thread except ImportError: import _dummy_thread as _thread _allocate_lock = _thread.allocate_lock _text_openflags = _os.O_RDWR | _os.O_CREAT | _os.O_EXCL -if hasattr(_os, 'O_CLOEXEC'): - _text_openflags |= _os.O_CLOEXEC -if hasattr(_os, 'O_NOINHERIT'): - _text_openflags |= _os.O_NOINHERIT if hasattr(_os, 'O_NOFOLLOW'): _text_openflags |= _os.O_NOFOLLOW @@ -89,8 +68,8 @@ else: # Fallback. All we need is something that raises OSError if the # file doesn't exist. def _stat(fn): - f = open(fn) - f.close() + fd = _os.open(fn, _os.O_RDONLY, cloexec=False) + os.close(fd) def _exists(fn): try: @@ -172,8 +151,8 @@ def _get_default_tempdir(): name = next(namer) filename = _os.path.join(dir, name) try: - fd = _os.open(filename, _bin_openflags, 0o600) - fp = _io.open(fd, 'wb') + fd = _os.open(filename, _bin_openflags, 0o600, cloexec=True) + fp = _io.open(fd, 'wb', cloexec=False) fp.write(b'blat') fp.close() _os.unlink(filename) @@ -210,8 +189,7 @@ def _mkstemp_inner(dir, pre, suf, flags) name = next(names) file = _os.path.join(dir, pre + name + suf) try: - fd = _os.open(file, flags, 0o600) - _set_cloexec(fd) + fd = _os.open(file, flags, 0o600, cloexec=True) return (fd, _os.path.abspath(file)) except FileExistsError: continue # try again @@ -431,7 +409,7 @@ def NamedTemporaryFile(mode='w+b', buffe (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags) file = _io.open(fd, mode, buffering=buffering, - newline=newline, encoding=encoding) + newline=newline, encoding=encoding, cloexec=False) return _TemporaryFileWrapper(file, name, delete) @@ -466,7 +444,7 @@ else: try: _os.unlink(name) return _io.open(fd, mode, buffering=buffering, - newline=newline, encoding=encoding) + newline=newline, encoding=encoding, cloexec=False) except: _os.close(fd) raise diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/test/mock_socket.py --- a/Lib/test/mock_socket.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/test/mock_socket.py Sat Jan 26 01:09:28 2013 +0100 @@ -102,7 +102,7 @@ class MockSocket: pass -def socket(family=None, type=None, proto=None): +def socket(family=None, type=None, proto=None, cloexec=True): return MockSocket() diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/test/test_asyncore.py --- a/Lib/test/test_asyncore.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/test/test_asyncore.py Sat Jan 26 01:09:28 2013 +0100 @@ -739,7 +739,7 @@ class BaseTestAPI: def test_create_socket(self): s = asyncore.dispatcher() - s.create_socket(self.family) + s.create_socket(self.family, cloexec=False) self.assertEqual(s.socket.family, self.family) SOCK_NONBLOCK = getattr(socket, 'SOCK_NONBLOCK', 0) self.assertEqual(s.socket.type, socket.SOCK_STREAM | SOCK_NONBLOCK) diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/test/test_builtin.py --- a/Lib/test/test_builtin.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/test/test_builtin.py Sat Jan 26 01:09:28 2013 +0100 @@ -943,29 +943,24 @@ class BuiltinTest(unittest.TestCase): def write_testfile(self): # NB the first 4 lines are also used to test input, below fp = open(TESTFN, 'w') - try: + self.addCleanup(unlink, TESTFN) + with fp: fp.write('1+1\n') fp.write('The quick brown fox jumps over the lazy dog') fp.write('.\n') fp.write('Dear John\n') fp.write('XXX'*100) fp.write('YYY'*100) - finally: - fp.close() def test_open(self): self.write_testfile() - fp = open(TESTFN, 'r') - try: + with open(TESTFN, 'r') as fp: self.assertEqual(fp.readline(4), '1+1\n') self.assertEqual(fp.readline(), 'The quick brown fox jumps over the lazy dog.\n') self.assertEqual(fp.readline(4), 'Dear') self.assertEqual(fp.readline(100), ' John\n') self.assertEqual(fp.read(300), 'XXX'*100) self.assertEqual(fp.read(1000), 'YYY'*100) - finally: - fp.close() - unlink(TESTFN) def test_open_default_encoding(self): old_environ = dict(os.environ) @@ -979,16 +974,23 @@ class BuiltinTest(unittest.TestCase): self.write_testfile() current_locale_encoding = locale.getpreferredencoding(False) - fp = open(TESTFN, 'w') - try: + with open(TESTFN, 'w') as fp: self.assertEqual(fp.encoding, current_locale_encoding) - finally: - fp.close() - unlink(TESTFN) finally: os.environ.clear() os.environ.update(old_environ) + @unittest.skipUnless(hasattr(os, "get_cloexec"), "need os.get_cloexec()") + def test_open_cloexec(self): + self.write_testfile() + + for cloexec in (None, False, True): + fileobj = open(TESTFN, cloexec=cloexec) + if cloexec is None: + cloexec = sys.getdefaultcloexec() + with fileobj: + self.assertEqual(os.get_cloexec(fileobj.fileno()), cloexec) + def test_ord(self): self.assertEqual(ord(' '), 32) self.assertEqual(ord('A'), 65) diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/test/test_bytes.py --- a/Lib/test/test_bytes.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/test/test_bytes.py Sat Jan 26 01:09:28 2013 +0100 @@ -700,7 +700,7 @@ class BytesTest(BaseBytesTest, unittest. type2test = bytes def test_buffer_is_readonly(self): - fd = os.dup(sys.stdin.fileno()) + fd = os.dup(sys.stdin.fileno(), cloexec=False) with open(fd, "rb", buffering=0) as f: self.assertRaises(TypeError, f.readinto, b"") diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/test/test_os.py --- a/Lib/test/test_os.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/test/test_os.py Sat Jan 26 01:09:28 2013 +0100 @@ -2201,6 +2201,91 @@ class OSErrorTests(unittest.TestCase): else: self.fail("No exception thrown by {}".format(func)) + +@unittest.skipUnless(hasattr(os, 'get_cloexec'), "need os.get_cloexec()") +class CloexecTests(unittest.TestCase): + def create_testfile(self): + fileobj = open(support.TESTFN, 'xb', cloexec=False) + self.addCleanup(support.unlink, support.TESTFN) + fileobj.close() + + def test_get_cloexec(self): + self.create_testfile() + + for cloexec in (False, True): + fd = os.open(support.TESTFN, os.O_RDONLY, cloexec=cloexec) + self.assertEqual(os.get_cloexec(fd), cloexec) + os.close(fd) + + def test_set_cloexec(self): + self.create_testfile() + + for cloexec in (False, True): + fd = os.open(support.TESTFN, os.O_RDONLY, cloexec=not cloexec) + os.set_cloexec(fd, cloexec) + self.assertEqual(os.get_cloexec(fd), cloexec) + os.close(fd) + + def test_open(self): + self.create_testfile() + + for cloexec in (None, False, True): + fd = os.open(support.TESTFN, os.O_RDONLY, cloexec=cloexec) + if cloexec is None: + cloexec = sys.getdefaultcloexec() + self.assertEqual(os.get_cloexec(fd), cloexec) + os.close(fd) + + @unittest.skipUnless(hasattr(os, 'pipe'), "need os.pipe()") + def test_pipe_cloexec(self): + for cloexec in (None, False, True): + rfd, wfd = os.pipe(cloexec=cloexec) + if cloexec is None: + cloexec = sys.getdefaultcloexec() + self.assertEqual(os.get_cloexec(rfd), cloexec) + self.assertEqual(os.get_cloexec(wfd), cloexec) + os.close(rfd) + os.close(wfd) + + @unittest.skipUnless(hasattr(os, 'dup'), "need os.dup()") + def test_dup_cloexec(self): + self.create_testfile() + + for cloexec in (None, False, True): + fd1 = os.open(support.TESTFN, os.O_RDONLY, cloexec=not cloexec) + fd2 = os.dup(fd1, cloexec=cloexec) + if cloexec is None: + cloexec = sys.getdefaultcloexec() + self.assertEqual(os.get_cloexec(fd2), cloexec) + os.close(fd1) + os.close(fd2) + + @unittest.skipUnless(hasattr(os, 'dup2'), "need os.dup2()") + def test_dup2_cloexec(self): + self.create_testfile() + + for cloexec in (None, False, True): + fd1 = os.open(support.TESTFN, os.O_RDONLY, cloexec=not cloexec) + fd2 = os.open(support.TESTFN, os.O_RDONLY, cloexec=not cloexec) + os.dup2(fd1, fd2, cloexec=cloexec) + if cloexec is None: + cloexec = sys.getdefaultcloexec() + self.assertEqual(os.get_cloexec(fd2), cloexec) + os.close(fd1) + os.close(fd2) + + @unittest.skipUnless(hasattr(os, 'openpty'), "need os.openpty()") + def test_openpty_cloexec(self): + for cloexec in (None, False, True): + master_fd, slave_fd = os.openpty(cloexec=cloexec) + if cloexec is None: + cloexec = sys.getdefaultcloexec() + self.assertEqual(os.get_cloexec(master_fd), cloexec) + self.assertEqual(os.get_cloexec(slave_fd), cloexec) + os.close(master_fd) + os.close(slave_fd) + + @support.reap_threads def test_main(): support.run_unittest( @@ -2231,6 +2316,7 @@ def test_main(): TermsizeTests, OSErrorTests, RemoveDirsTests, + CloexecTests, ) if __name__ == "__main__": diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/test/test_shutil.py --- a/Lib/test/test_shutil.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/test/test_shutil.py Sat Jan 26 01:09:28 2013 +0100 @@ -1527,7 +1527,7 @@ class TestCopyFile(unittest.TestCase): self._delete = True def test_w_source_open_fails(self): - def _open(filename, mode='r'): + def _open(filename, mode='r', cloexec=True): if filename == 'srcfile': raise OSError('Cannot open "srcfile"') assert 0 # shouldn't reach here. @@ -1540,7 +1540,7 @@ class TestCopyFile(unittest.TestCase): srcfile = self.Faux() - def _open(filename, mode='r'): + def _open(filename, mode='r', cloexec=True): if filename == 'srcfile': return srcfile if filename == 'destfile': @@ -1560,7 +1560,7 @@ class TestCopyFile(unittest.TestCase): srcfile = self.Faux() destfile = self.Faux(True) - def _open(filename, mode='r'): + def _open(filename, mode='r', cloexec=True): if filename == 'srcfile': return srcfile if filename == 'destfile': @@ -1582,7 +1582,7 @@ class TestCopyFile(unittest.TestCase): srcfile = self.Faux(True) destfile = self.Faux() - def _open(filename, mode='r'): + def _open(filename, mode='r', cloexec=True): if filename == 'srcfile': return srcfile if filename == 'destfile': diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/test/test_socket.py --- a/Lib/test/test_socket.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/test/test_socket.py Sat Jan 26 01:09:28 2013 +0100 @@ -24,10 +24,6 @@ import math import pickle import struct try: - import fcntl -except ImportError: - fcntl = False -try: import multiprocessing except ImportError: multiprocessing = False @@ -1073,7 +1069,7 @@ class GeneralModuleTests(unittest.TestCa def testNewAttributes(self): # testing .family, .type and .protocol - sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, cloexec=False) self.assertEqual(sock.family, socket.AF_INET) self.assertEqual(sock.type, socket.SOCK_STREAM) self.assertEqual(sock.proto, 0) @@ -4653,16 +4649,44 @@ class ContextManagersTest(ThreadedTCPSoc self.assertRaises(OSError, sock.sendall, b'foo') -@unittest.skipUnless(hasattr(socket, "SOCK_CLOEXEC"), - "SOCK_CLOEXEC not defined") -@unittest.skipUnless(fcntl, "module fcntl not available") -class CloexecConstantTest(unittest.TestCase): +@unittest.skipUnless(hasattr(os, 'get_cloexec'), "need os.get_cloexec()") +class CloexecTest(unittest.TestCase): + @unittest.skipUnless(hasattr(socket, "SOCK_CLOEXEC"), + "SOCK_CLOEXEC not defined") @support.requires_linux_version(2, 6, 28) def test_SOCK_CLOEXEC(self): with socket.socket(socket.AF_INET, socket.SOCK_STREAM | socket.SOCK_CLOEXEC) as s: self.assertTrue(s.type & socket.SOCK_CLOEXEC) - self.assertTrue(fcntl.fcntl(s, fcntl.F_GETFD) & fcntl.FD_CLOEXEC) + self.assertTrue(os.get_cloexec(sock.fileno())) + + def test_socket_cloexec(self): + for cloexec in (None, False, True): + sock = socket.socket(cloexec=cloexec) + if cloexec is None: + cloexec = sys.getdefaultcloexec() + with sock: + self.assertEqual(os.get_cloexec(sock.fileno()), cloexec) + + def test_socket_set_cloexec(self): + sock = socket.socket(cloexec=False) + with sock: + # On Windows, fileno() returns an handle, not a file descriptor. + # Check if it is accepted. + os.set_cloexec(sock.fileno()) + self.assertEqual(os.get_cloexec(sock.fileno()), True) + + @unittest.skipUnless(hasattr(socket, "socketpair"), + "need socket.socketpair()") + def test_socketpair_cloexec(self): + for cloexec in (None, False, True): + s1, s2 = socket.socketpair(cloexec=cloexec) + if cloexec is None: + cloexec = sys.getdefaultcloexec() + self.assertEqual(os.get_cloexec(s1.fileno()), cloexec) + self.assertEqual(os.get_cloexec(s2.fileno()), cloexec) + s1.close() + s2.close() @unittest.skipUnless(hasattr(socket, "SOCK_NONBLOCK"), @@ -4831,7 +4855,7 @@ def test_main(): NetworkConnectionAttributesTest, NetworkConnectionBehaviourTest, ContextManagersTest, - CloexecConstantTest, + CloexecTest, NonblockConstantTest ]) if hasattr(socket, "socketpair"): diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/test/test_subprocess.py --- a/Lib/test/test_subprocess.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/test/test_subprocess.py Sat Jan 26 01:09:28 2013 +0100 @@ -1438,7 +1438,7 @@ class POSIXProcessTestCase(BaseTestCase) self.assertEqual((out, err), (b'apple', b'orange')) finally: for b, a in zip(newfds, fds): - os.dup2(b, a) + os.dup2(b, a, cloexec=False) for b in newfds: os.close(b) @@ -1498,7 +1498,7 @@ class POSIXProcessTestCase(BaseTestCase) finally: # restore the original fd's underneath sys.stdin, etc. for std, saved in enumerate(saved_fds): - os.dup2(saved, std) + os.dup2(saved, std, cloexec=False) os.close(saved) for fd in temp_fds: @@ -1550,7 +1550,7 @@ class POSIXProcessTestCase(BaseTestCase) err = support.strip_python_stderr(os.read(stderr_no, 1024)) finally: for std, saved in enumerate(saved_fds): - os.dup2(saved, std) + os.dup2(saved, std, cloexec=False) os.close(saved) self.assertEqual(out, b"got STDIN") @@ -1711,14 +1711,14 @@ class POSIXProcessTestCase(BaseTestCase) def test_close_fds(self): fd_status = support.findfile("fd_status.py", subdir="subprocessdata") - fds = os.pipe() + fds = os.pipe(cloexec=False) self.addCleanup(os.close, fds[0]) self.addCleanup(os.close, fds[1]) open_fds = set(fds) # add a bunch more fds for _ in range(9): - fd = os.open("/dev/null", os.O_RDONLY) + fd = os.open("/dev/null", os.O_RDONLY, cloexec=False) self.addCleanup(os.close, fd) open_fds.add(fd) @@ -1763,7 +1763,7 @@ class POSIXProcessTestCase(BaseTestCase) open_fds = set() for x in range(5): - fds = os.pipe() + fds = os.pipe(cloexec=False) self.addCleanup(os.close, fds[0]) self.addCleanup(os.close, fds[1]) open_fds.update(fds) diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/test/test_sys.py --- a/Lib/test/test_sys.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/test/test_sys.py Sat Jan 26 01:09:28 2013 +0100 @@ -8,6 +8,7 @@ import operator import codecs import gc import sysconfig +from test.script_helper import assert_python_ok # count the number of test runs, used to create unique # strings to intern in test_intern() @@ -608,7 +609,6 @@ class SysModuleTest(unittest.TestCase): def test_debugmallocstats(self): # Test sys._debugmallocstats() - from test.script_helper import assert_python_ok args = ['-c', 'import sys; sys._debugmallocstats()'] ret, out, err = assert_python_ok(*args) self.assertIn(b"free PyDictObjects", err) @@ -642,6 +642,30 @@ class SysModuleTest(unittest.TestCase): c = sys.getallocatedblocks() self.assertIn(c, range(b - 50, b + 50)) + def test_getdefaultcloexec_cmdline(self): + code = 'import sys; print(sys.getdefaultcloexec())' + + ok, stdout, stderr = assert_python_ok('-c', code) + self.assertEqual(stdout.rstrip(), b'False') + + ok, stdout, stderr = assert_python_ok('-c', code, PYTHONCLOEXEC='1') + self.assertEqual(stdout.rstrip(), b'True') + + ok, stdout, stderr = assert_python_ok('-E', '-c', code, PYTHONCLOEXEC='1') + self.assertEqual(stdout.rstrip(), b'False') + + ok, stdout, stderr = assert_python_ok('-e', '-c', code) + self.assertEqual(stdout.rstrip(), b'True') + + def test_setdefaultcloexec(self): + code = '\n'.join(( + 'import sys', + 'sys.setdefaultcloexec()', + 'print(sys.getdefaultcloexec())', + )) + ok, stdout, stderr = assert_python_ok('-c', code) + self.assertEqual(stdout.rstrip(), b'True') + class SizeofTest(unittest.TestCase): diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/urllib/request.py --- a/Lib/urllib/request.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/urllib/request.py Sat Jan 26 01:09:28 2013 +0100 @@ -193,7 +193,7 @@ def urlretrieve(url, filename=None, repo # Handle temporary file setup. if filename: - tfp = open(filename, 'wb') + tfp = open(filename, 'wb', cloexec=True) else: tfp = tempfile.NamedTemporaryFile(delete=False) filename = tfp.name @@ -1443,7 +1443,7 @@ class FileHandler(BaseHandler): origurl = 'file://' + host + filename else: origurl = 'file://' + filename - return addinfourl(open(localfile, 'rb'), headers, origurl) + return addinfourl(open(localfile, 'rb', cloexec=True), headers, origurl) except OSError as exp: # users shouldn't expect OSErrors coming from urlopen() raise URLError(exp) @@ -1694,7 +1694,7 @@ class URLopener: fullurl = quote(fullurl, safe="%/:=&?~#+!$,;'@()*[]|") if self.tempcache and fullurl in self.tempcache: filename, headers = self.tempcache[fullurl] - fp = open(filename, 'rb') + fp = open(filename, 'rb', cloexec=True) return addinfourl(fp, headers, fullurl) urltype, url = splittype(fullurl) if not urltype: @@ -1754,7 +1754,7 @@ class URLopener: try: headers = fp.info() if filename: - tfp = open(filename, 'wb') + tfp = open(filename, 'wb', cloexec=True) else: import tempfile garbage, path = splittype(url) @@ -1764,7 +1764,7 @@ class URLopener: suffix = os.path.splitext(path)[1] (fd, filename) = tempfile.mkstemp(suffix) self.__tempfiles.append(filename) - tfp = os.fdopen(fd, 'wb') + tfp = os.fdopen(fd, 'wb', cloexec=False) try: result = filename, headers if self.tempcache is not None: @@ -1957,7 +1957,7 @@ class URLopener: urlfile = file if file[:1] == '/': urlfile = 'file://' + file - return addinfourl(open(localname, 'rb'), headers, urlfile) + return addinfourl(open(localname, 'rb', cloexec=True), headers, urlfile) host, port = splitport(host) if (not port and socket.gethostbyname(host) in ((localhost(),) + thishost())): @@ -1966,7 +1966,7 @@ class URLopener: urlfile = 'file://' + file elif file[:2] == './': raise ValueError("local file url may start with / or file:. Unknown url of type: %s" % url) - return addinfourl(open(localname, 'rb'), headers, urlfile) + return addinfourl(open(localname, 'rb', cloexec=True), headers, urlfile) raise URLError('local file error: not on local host') def open_ftp(self, url): diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/venv/__init__.py --- a/Lib/venv/__init__.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/venv/__init__.py Sat Jan 26 01:09:28 2013 +0100 @@ -157,7 +157,7 @@ class EnvBuilder: being processed. """ context.cfg_path = path = os.path.join(context.env_dir, 'pyvenv.cfg') - with open(path, 'w', encoding='utf-8') as f: + with open(path, 'w', encoding='utf-8', cloexec=True) as f: f.write('home = %s\n' % context.python_dir) if self.system_site_packages: incl = 'true' @@ -309,7 +309,7 @@ class EnvBuilder: if not os.path.exists(dstdir): os.makedirs(dstdir) dstfile = os.path.join(dstdir, f) - with open(srcfile, 'rb') as f: + with open(srcfile, 'rb', cloexec=True) as f: data = f.read() if srcfile.endswith('.exe'): mode = 'wb' @@ -323,7 +323,7 @@ class EnvBuilder: logger.warning('unable to copy script %r, ' 'may be binary: %s', srcfile, e) if data is not None: - with open(dstfile, mode) as f: + with open(dstfile, mode, cloexec=True) as f: f.write(data) shutil.copymode(srcfile, dstfile) diff -r f87d85f7596c -r 9bdfa1a3ea8c Lib/xmlrpc/server.py --- a/Lib/xmlrpc/server.py Wed Jan 16 15:19:16 2013 +0100 +++ b/Lib/xmlrpc/server.py Sat Jan 26 01:09:28 2013 +0100 @@ -587,10 +587,8 @@ class SimpleXMLRPCServer(socketserver.TC # [Bug #1222790] If possible, set close-on-exec flag; if a # method spawns a subprocess, the subprocess shouldn't have # the listening socket open. - if fcntl is not None and hasattr(fcntl, 'FD_CLOEXEC'): - flags = fcntl.fcntl(self.fileno(), fcntl.F_GETFD) - flags |= fcntl.FD_CLOEXEC - fcntl.fcntl(self.fileno(), fcntl.F_SETFD, flags) + if hasattr(os, 'set_cloexec'): + os.set_cloexec(self.fileno()) class MultiPathXMLRPCServer(SimpleXMLRPCServer): """Multipath XML-RPC Server diff -r f87d85f7596c -r 9bdfa1a3ea8c Modules/_cursesmodule.c --- a/Modules/_cursesmodule.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Modules/_cursesmodule.c Sat Jan 26 01:09:28 2013 +0100 @@ -1701,6 +1701,7 @@ PyCursesWindow_PutWin(PyCursesWindowObje fd = mkstemp(fn); if (fd < 0) return PyErr_SetFromErrnoWithFilename(PyExc_IOError, fn); + _Py_try_set_cloexec(fd, 1); fp = fdopen(fd, "wb+"); if (fp == NULL) { close(fd); @@ -2264,6 +2265,7 @@ PyCurses_GetWin(PyCursesWindowObject *se fd = mkstemp(fn); if (fd < 0) return PyErr_SetFromErrnoWithFilename(PyExc_IOError, fn); + _Py_try_set_cloexec(fd, 1); fp = fdopen(fd, "wb+"); if (fp == NULL) { close(fd); diff -r f87d85f7596c -r 9bdfa1a3ea8c Modules/_freeze_importlib.c --- a/Modules/_freeze_importlib.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Modules/_freeze_importlib.c Sat Jan 26 01:09:28 2013 +0100 @@ -12,6 +12,10 @@ #include #endif +#ifdef HAVE_FCNTL_H +#include +#endif /* HAVE_FCNTL_H */ + /* To avoid a circular dependency on frozen.o, we create our own structure of frozen modules instead, left deliberately blank so as to avoid @@ -33,6 +37,7 @@ const char header[] = "/* Auto-generated int main(int argc, char *argv[]) { + int fd; char *inpath, *outpath; FILE *infile, *outfile = NULL; struct stat st; @@ -49,7 +54,11 @@ main(int argc, char *argv[]) } inpath = argv[1]; outpath = argv[2]; - infile = fopen(inpath, "rb"); + fd = _Py_open(inpath, O_RDWR); + if (fd >= 0) + infile = fdopen(fd, "rb"); + else + infile = NULL; if (infile == NULL) { fprintf(stderr, "cannot open '%s' for reading\n", inpath); return 1; @@ -99,7 +108,11 @@ main(int argc, char *argv[]) /* Open the file in text mode. The hg checkout should be using the eol extension, which in turn should cause the EOL style match the C library's text mode */ - outfile = fopen(outpath, "w"); + fd = _Py_open(outpath, O_WRONLY); + if (fd >= 0) + outfile = fdopen(fd, "w"); + else + outfile = NULL; if (outfile == NULL) { fprintf(stderr, "cannot open '%s' for writing\n", outpath); return 1; diff -r f87d85f7596c -r 9bdfa1a3ea8c Modules/_io/_iomodule.c --- a/Modules/_io/_iomodule.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Modules/_io/_iomodule.c Sat Jan 26 01:09:28 2013 +0100 @@ -94,8 +94,8 @@ PyDoc_STRVAR(module_doc, * The main open() function */ PyDoc_STRVAR(open_doc, -"open(file, mode='r', buffering=-1, encoding=None,\n" -" errors=None, newline=None, closefd=True, opener=None) -> file object\n" +"open(file, mode='r', buffering=-1, encoding=None, errors=None,\n" +" newline=None, closefd=True, opener=None, cloexec=None) -> file object\n" "\n" "Open file and return a stream. Raise IOError upon failure.\n" "\n" @@ -199,6 +199,8 @@ PyDoc_STRVAR(open_doc, "file descriptor (passing os.open as *opener* results in functionality\n" "similar to passing None).\n" "\n" +"If cloexec is True, the close-on-exec flag is set.\n" +"\n" "open() returns a file object whose type depends on the mode, and\n" "through which the standard file operations such as reading and writing\n" "are performed. When open() is used to open a file in a text mode ('w',\n" @@ -219,8 +221,9 @@ io_open(PyObject *self, PyObject *args, { char *kwlist[] = {"file", "mode", "buffering", "encoding", "errors", "newline", - "closefd", "opener", NULL}; + "closefd", "opener", "cloexec", NULL}; PyObject *file, *opener = Py_None; + int cloexec = Py_DefaultCloexec; char *mode = "r"; int buffering = -1, closefd = 1; char *encoding = NULL, *errors = NULL, *newline = NULL; @@ -238,10 +241,11 @@ io_open(PyObject *self, PyObject *args, _Py_IDENTIFIER(fileno); _Py_IDENTIFIER(mode); - if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|sizzziO:open", kwlist, + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|sizzziOO&:open", kwlist, &file, &mode, &buffering, &encoding, &errors, &newline, - &closefd, &opener)) { + &closefd, &opener, + _Py_cloexec_converter, &cloexec)) { return NULL; } @@ -345,7 +349,7 @@ io_open(PyObject *self, PyObject *args, /* Create the Raw file stream */ raw = PyObject_CallFunction((PyObject *)&PyFileIO_Type, - "OsiO", file, rawmode, closefd, opener); + "OsiOi", file, rawmode, closefd, opener, cloexec); if (raw == NULL) return NULL; diff -r f87d85f7596c -r 9bdfa1a3ea8c Modules/_io/fileio.c --- a/Modules/_io/fileio.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Modules/_io/fileio.c Sat Jan 26 01:09:28 2013 +0100 @@ -202,14 +202,14 @@ check_fd(int fd) return 0; } - static int fileio_init(PyObject *oself, PyObject *args, PyObject *kwds) { fileio *self = (fileio *) oself; - static char *kwlist[] = {"file", "mode", "closefd", "opener", NULL}; + static char *kwlist[] = {"file", "mode", "closefd", "opener", "cloexec", NULL}; const char *name = NULL; PyObject *nameobj, *stringobj = NULL, *opener = Py_None; + int cloexec = Py_DefaultCloexec; char *mode = "r"; char *s; #ifdef MS_WINDOWS @@ -221,6 +221,12 @@ fileio_init(PyObject *oself, PyObject *a int fd = -1; int closefd = 1; int fd_is_own = 0; +#ifdef O_CLOEXEC + static int cloexec_works = -1; + int *atomic_flags_works = &cloexec_works; +#else + int *atomic_flags_works = NULL; +#endif assert(PyFileIO_Check(oself)); if (self->fd >= 0) { @@ -233,9 +239,10 @@ fileio_init(PyObject *oself, PyObject *a self->fd = -1; } - if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|siO:fileio", + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|siOO&:fileio", kwlist, &nameobj, &mode, &closefd, - &opener)) + &opener, + _Py_cloexec_converter, &cloexec)) return -1; if (PyFloat_Check(nameobj)) { @@ -345,6 +352,13 @@ fileio_init(PyObject *oself, PyObject *a if (append) flags |= O_APPEND; #endif + if (cloexec) { +#ifdef MS_WINDOWS + flags |= O_NOINHERIT; +#elif defined(O_CLOEXEC) + flags |= O_CLOEXEC; +#endif + } if (fd >= 0) { if (check_fd(fd)) @@ -361,7 +375,7 @@ fileio_init(PyObject *oself, PyObject *a } errno = 0; - if (opener == Py_None) { + if (opener == Py_None && !cloexec) { Py_BEGIN_ALLOW_THREADS #ifdef MS_WINDOWS if (widename != NULL) @@ -369,10 +383,24 @@ fileio_init(PyObject *oself, PyObject *a else #endif self->fd = open(name, flags, 0666); + Py_END_ALLOW_THREADS - } else { - PyObject *fdobj = PyObject_CallFunction( - opener, "Oi", nameobj, flags); + } + else if (opener == Py_None && cloexec) { + /* We hold the GIL which offers some protection from other code + * calling fork() before the CLOEXEC flags have been set but we + * can't guarantee anything without O_CLOEXEC / O_NOINHERIT. */ +#ifdef MS_WINDOWS + if (widename != NULL) + self->fd = _wopen(widename, flags, 0666); + else +#endif + self->fd = open(name, flags, 0666); + } + else { + PyObject *fdobj; + + fdobj = PyObject_CallFunction(opener, "Oi", nameobj, flags); if (fdobj == NULL) goto error; if (!PyLong_Check(fdobj)) { @@ -389,6 +417,15 @@ fileio_init(PyObject *oself, PyObject *a } } +#ifndef MS_WINDOWS + if (cloexec && self->fd >= 0) { + if (_Py_set_cloexec(self->fd, 1, atomic_flags_works) < 0) { + close(self->fd); + self->fd = -1; + } + } +#endif + fd_is_own = 1; if (self->fd < 0) { PyErr_SetFromErrnoWithFilenameObject(PyExc_OSError, nameobj); diff -r f87d85f7596c -r 9bdfa1a3ea8c Modules/_posixsubprocess.c --- a/Modules/_posixsubprocess.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Modules/_posixsubprocess.c Sat Jan 26 01:09:28 2013 +0100 @@ -35,7 +35,7 @@ # define FD_DIR "/proc/self/fd" #endif -#define POSIX_CALL(call) if ((call) == -1) goto error +#define POSIX_CALL(call) do { if ((call) == -1) goto error; } while (0) /* Maximum file descriptor, initialized on module load. */ @@ -87,7 +87,7 @@ static int if (stat("/dev", &dev_stat) != 0) return 0; if (stat(FD_DIR, &dev_fd_stat) != 0) - return 0; + return 0; if (dev_stat.st_dev == dev_fd_stat.st_dev) return 0; /* / == /dev == /dev/fd means it is static. #fail */ return 1; @@ -211,18 +211,8 @@ static void int fd_dir_fd; if (start_fd >= end_fd) return; -#ifdef O_CLOEXEC - fd_dir_fd = open(FD_DIR, O_RDONLY | O_CLOEXEC, 0); -#else - fd_dir_fd = open(FD_DIR, O_RDONLY, 0); -#ifdef FD_CLOEXEC - { - int old = fcntl(fd_dir_fd, F_GETFD); - if (old != -1) - fcntl(fd_dir_fd, F_SETFD, old | FD_CLOEXEC); - } -#endif -#endif + + fd_dir_fd = _Py_open_cloexec(FD_DIR, O_RDONLY); if (fd_dir_fd == -1) { /* No way to get a list of open fds. */ _close_fds_by_brute_force(start_fd, end_fd, py_fds_to_keep); @@ -363,15 +353,12 @@ child_exec(char *const exec_array[], char hex_errno[sizeof(saved_errno)*2+1]; /* Close parent's pipe ends. */ - if (p2cwrite != -1) { + if (p2cwrite != -1) POSIX_CALL(close(p2cwrite)); - } - if (c2pread != -1) { + if (c2pread != -1) POSIX_CALL(close(c2pread)); - } - if (errread != -1) { + if (errread != -1) POSIX_CALL(close(errread)); - } POSIX_CALL(close(errpipe_read)); /* When duping fds, if there arises a situation where one of the fds is @@ -384,39 +371,27 @@ child_exec(char *const exec_array[], /* Dup fds for child. dup2() removes the CLOEXEC flag but we must do it ourselves if dup2() would be a no-op (issue #10806). */ - if (p2cread == 0) { - int old = fcntl(p2cread, F_GETFD); - if (old != -1) - fcntl(p2cread, F_SETFD, old & ~FD_CLOEXEC); - } else if (p2cread != -1) { + if (p2cread == 0) + _Py_try_set_cloexec(p2cread, 0); + else if (p2cread != -1) POSIX_CALL(dup2(p2cread, 0)); /* stdin */ - } - if (c2pwrite == 1) { - int old = fcntl(c2pwrite, F_GETFD); - if (old != -1) - fcntl(c2pwrite, F_SETFD, old & ~FD_CLOEXEC); - } else if (c2pwrite != -1) { + if (c2pwrite == 1) + _Py_try_set_cloexec(c2pwrite, 0); + else if (c2pwrite != -1) POSIX_CALL(dup2(c2pwrite, 1)); /* stdout */ - } - if (errwrite == 2) { - int old = fcntl(errwrite, F_GETFD); - if (old != -1) - fcntl(errwrite, F_SETFD, old & ~FD_CLOEXEC); - } else if (errwrite != -1) { + if (errwrite == 2) + _Py_try_set_cloexec(errwrite, 0); + else if (errwrite != -1) POSIX_CALL(dup2(errwrite, 2)); /* stderr */ - } /* Close pipe fds. Make sure we don't close the same fd more than */ /* once, or standard fds. */ - if (p2cread > 2) { + if (p2cread > 2) POSIX_CALL(close(p2cread)); - } - if (c2pwrite > 2 && c2pwrite != p2cread) { + if (c2pwrite > 2 && c2pwrite != p2cread) POSIX_CALL(close(c2pwrite)); - } - if (errwrite != c2pwrite && errwrite != p2cread && errwrite > 2) { + if (errwrite != c2pwrite && errwrite != p2cread && errwrite > 2) POSIX_CALL(close(errwrite)); - } if (close_fds) { int local_max_fd = max_fd; @@ -549,7 +524,7 @@ subprocess_fork_exec(PyObject* self, PyO PyObject *result; _Py_IDENTIFIER(isenabled); _Py_IDENTIFIER(disable); - + gc_module = PyImport_ImportModule("gc"); if (gc_module == NULL) return NULL; @@ -726,52 +701,6 @@ Returns: the child process's PID.\n\ Raises: Only on an error in the parent process.\n\ "); -PyDoc_STRVAR(subprocess_cloexec_pipe_doc, -"cloexec_pipe() -> (read_end, write_end)\n\n\ -Create a pipe whose ends have the cloexec flag set."); - -static PyObject * -subprocess_cloexec_pipe(PyObject *self, PyObject *noargs) -{ - int fds[2]; - int res; -#ifdef HAVE_PIPE2 - Py_BEGIN_ALLOW_THREADS - res = pipe2(fds, O_CLOEXEC); - Py_END_ALLOW_THREADS - if (res != 0 && errno == ENOSYS) - { - { -#endif - /* We hold the GIL which offers some protection from other code calling - * fork() before the CLOEXEC flags have been set but we can't guarantee - * anything without pipe2(). */ - long oldflags; - - res = pipe(fds); - - if (res == 0) { - oldflags = fcntl(fds[0], F_GETFD, 0); - if (oldflags < 0) res = oldflags; - } - if (res == 0) - res = fcntl(fds[0], F_SETFD, oldflags | FD_CLOEXEC); - - if (res == 0) { - oldflags = fcntl(fds[1], F_GETFD, 0); - if (oldflags < 0) res = oldflags; - } - if (res == 0) - res = fcntl(fds[1], F_SETFD, oldflags | FD_CLOEXEC); -#ifdef HAVE_PIPE2 - } - } -#endif - if (res != 0) - return PyErr_SetFromErrno(PyExc_OSError); - return Py_BuildValue("(ii)", fds[0], fds[1]); -} - /* module level code ********************************************************/ PyDoc_STRVAR(module_doc, @@ -780,7 +709,6 @@ PyDoc_STRVAR(module_doc, static PyMethodDef module_methods[] = { {"fork_exec", subprocess_fork_exec, METH_VARARGS, subprocess_fork_exec_doc}, - {"cloexec_pipe", subprocess_cloexec_pipe, METH_NOARGS, subprocess_cloexec_pipe_doc}, {NULL, NULL} /* sentinel */ }; diff -r f87d85f7596c -r 9bdfa1a3ea8c Modules/main.c --- a/Modules/main.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Modules/main.c Sat Jan 26 01:09:28 2013 +0100 @@ -22,6 +22,10 @@ #include #endif +#ifdef HAVE_FCNTL_H +#include +#endif /* HAVE_FCNTL_H */ + #if defined(MS_WINDOWS) #define PYTHONHOMEHELP "\\lib" #else @@ -43,7 +47,7 @@ static wchar_t **orig_argv; static int orig_argc; /* command line options */ -#define BASE_OPTS L"bBc:dEhiJm:OqRsStuvVW:xX:?" +#define BASE_OPTS L"bBc:deEhiJm:OqRsStuvVW:xX:?" #define PROGRAM_OPTS BASE_OPTS @@ -59,6 +63,8 @@ Options and arguments (and corresponding -B : don't write .py[co] files on import; also PYTHONDONTWRITEBYTECODE=x\n\ -c cmd : program passed in as string (terminates option list)\n\ -d : debug output from parser; also PYTHONDEBUG=x\n\ +-e : set default value of the cloexec (close-on-exec) parameter\n\ + to True; also PYTHONCLOEXEC=1\n\ -E : ignore PYTHON* environment variables (such as PYTHONPATH)\n\ -h : print this help message and exit (also --help)\n\ "; @@ -104,6 +110,8 @@ PYTHONHASHSEED: if this variable is set to seed the hashes of str, bytes and datetime objects. It can also be\n\ set to an integer in the range [0,4294967295] to get hash values with a\n\ predictable seed.\n\ +PYTHONCLOEXEC: set default value of the cloexec (close-on-exec) parameter\n\ + to True.\n\ "; static int @@ -141,7 +149,12 @@ static void RunStartupFile(PyCompilerFla { char *startup = Py_GETENV("PYTHONSTARTUP"); if (startup != NULL && startup[0] != '\0') { - FILE *fp = fopen(startup, "r"); + FILE *fp; + int fd = _Py_open(startup, O_RDONLY); + if (fd >= 0) + fp = fdopen(fd, "r"); + else + fp = NULL; if (fp != NULL) { (void) PyRun_SimpleFileExFlags(fp, startup, 0, cf); PyErr_Clear(); @@ -403,6 +416,10 @@ Py_Main(int argc, wchar_t **argv) Py_DontWriteBytecodeFlag++; break; + case 'e': + Py_DefaultCloexec = 1; + break; + case 's': Py_NoUserSiteDirectory++; break; diff -r f87d85f7596c -r 9bdfa1a3ea8c Modules/mmapmodule.c --- a/Modules/mmapmodule.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Modules/mmapmodule.c Sat Jan 26 01:09:28 2013 +0100 @@ -1205,7 +1205,7 @@ new_mmap_object(PyTypeObject *type, PyOb flags |= MAP_ANONYMOUS; #else /* SVR4 method to map anonymous memory is to open /dev/zero */ - fd = devzero = open("/dev/zero", O_RDWR); + fd = devzero = _Py_open("/dev/zero", O_RDWR); if (devzero == -1) { Py_DECREF(m_obj); PyErr_SetFromErrno(PyExc_OSError); @@ -1219,6 +1219,7 @@ new_mmap_object(PyTypeObject *type, PyOb PyErr_SetFromErrno(PyExc_OSError); return NULL; } + _Py_try_set_default_cloexec(m_obj->fd); } m_obj->data = mmap(NULL, map_size, diff -r f87d85f7596c -r 9bdfa1a3ea8c Modules/ossaudiodev.c --- a/Modules/ossaudiodev.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Modules/ossaudiodev.c Sat Jan 26 01:09:28 2013 +0100 @@ -115,7 +115,9 @@ newossobject(PyObject *arg) one open at a time. This does *not* affect later I/O; OSS provides a special ioctl() for non-blocking read/write, which is exposed via oss_nonblock() below. */ - if ((fd = open(devicename, imode|O_NONBLOCK)) == -1) { + fd = _Py_open(devicename, imode|O_NONBLOCK); + + if (fd == -1) { PyErr_SetFromErrnoWithFilename(PyExc_IOError, devicename); return NULL; } @@ -177,7 +179,8 @@ newossmixerobject(PyObject *arg) devicename = "/dev/mixer"; } - if ((fd = open(devicename, O_RDWR)) == -1) { + fd = _Py_open(devicename, O_RDWR); + if (fd == -1) { PyErr_SetFromErrnoWithFilename(PyExc_IOError, devicename); return NULL; } diff -r f87d85f7596c -r 9bdfa1a3ea8c Modules/posixmodule.c --- a/Modules/posixmodule.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Modules/posixmodule.c Sat Jan 26 01:09:28 2013 +0100 @@ -5447,14 +5447,16 @@ error: #if defined(HAVE_OPENPTY) || defined(HAVE__GETPTY) || defined(HAVE_DEV_PTMX) PyDoc_STRVAR(posix_openpty__doc__, -"openpty() -> (master_fd, slave_fd)\n\n\ +"openpty(cloexec=None) -> (master_fd, slave_fd)\n\n\ Open a pseudo-terminal, returning open fd's for both master and slave end.\n"); static PyObject * -posix_openpty(PyObject *self, PyObject *noargs) -{ +posix_openpty(PyObject *self, PyObject *args, PyObject *kwds) +{ + static char *keywords[] = {"cloexec", NULL}; int master_fd, slave_fd; #ifndef HAVE_OPENPTY + int flags; char * slave_name; #endif #if defined(HAVE_DEV_PTMX) && !defined(HAVE_OPENPTY) && !defined(HAVE__GETPTY) @@ -5463,22 +5465,51 @@ posix_openpty(PyObject *self, PyObject * extern char *ptsname(int fildes); #endif #endif + int cloexec = Py_DefaultCloexec; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "|O&:openpty", keywords, + _Py_cloexec_converter, &cloexec)) + return NULL; #ifdef HAVE_OPENPTY if (openpty(&master_fd, &slave_fd, NULL, NULL, NULL) != 0) return posix_error(); + + if (cloexec && _Py_set_cloexec(master_fd, 1, NULL) < 0) + goto error; + if (cloexec && _Py_set_cloexec(slave_fd, 1, NULL) < 0) + goto error; + #elif defined(HAVE__GETPTY) slave_name = _getpty(&master_fd, O_RDWR, 0666, 0); if (slave_name == NULL) return posix_error(); - - slave_fd = open(slave_name, O_RDWR); + if (cloexec && _Py_set_cloexec(master_fd, 1, NULL) < 0) + goto error; + + flags = O_RDWR; +#ifdef O_CLOEXEC + if (cloexec) + flags |= O_CLOEXEC; +#endif + slave_fd = open(slave_name, flags); if (slave_fd < 0) return posix_error(); -#else - master_fd = open(DEV_PTY_FILE, O_RDWR | O_NOCTTY); /* open master */ + if (cloexec && _Py_set_cloexec(slave_fd, 1, NULL) < 0) + goto error; + +#else + flags = O_RDWR | O_NOCTTY; +#ifdef O_CLOEXEC + if (cloexec) + flags |= O_CLOEXEC; +#endif + master_fd = open(DEV_PTY_FILE, flags); /* open master */ if (master_fd < 0) return posix_error(); + if (cloexec && _Py_set_cloexec(master_fd, 1, NULL) < 0) + goto error; + sig_saved = PyOS_setsig(SIGCHLD, SIG_DFL); /* change permission of slave */ if (grantpt(master_fd) < 0) { @@ -5494,9 +5525,16 @@ posix_openpty(PyObject *self, PyObject * slave_name = ptsname(master_fd); /* get name of slave */ if (slave_name == NULL) return posix_error(); - slave_fd = open(slave_name, O_RDWR | O_NOCTTY); /* open slave */ + flags = O_RDWR | O_NOCTTY; +#ifdef O_CLOEXEC + if (cloexec) + flags |= O_CLOEXEC; +#endif + slave_fd = open(slave_name, flags); /* open slave */ if (slave_fd < 0) return posix_error(); + if (cloexec && _Py_set_cloexec(slave_fd, 1, NULL) < 0) + goto error; #if !defined(__CYGWIN__) && !defined(HAVE_DEV_PTC) ioctl(slave_fd, I_PUSH, "ptem"); /* push ptem */ ioctl(slave_fd, I_PUSH, "ldterm"); /* push ldterm */ @@ -5508,6 +5546,10 @@ posix_openpty(PyObject *self, PyObject * return Py_BuildValue("(ii)", master_fd, slave_fd); +error: + close(master_fd); + close(slave_fd); + return NULL; } #endif /* defined(HAVE_OPENPTY) || defined(HAVE__GETPTY) || defined(HAVE_DEV_PTMX) */ @@ -6993,13 +7035,15 @@ posix_tcsetpgrp(PyObject *self, PyObject /* Functions acting on file descriptors */ PyDoc_STRVAR(posix_open__doc__, -"open(path, flags, mode=0o777, *, dir_fd=None)\n\n\ +"open(path, flags, mode=0o777, *, dir_fd=None, cloexec=None)\n\n\ Open a file for low level IO. Returns a file handle (integer).\n\ \n\ If dir_fd is not None, it should be a file descriptor open to a directory,\n\ and path should be relative; path will then be relative to that directory.\n\ dir_fd may not be implemented on your platform.\n\ - If it is unavailable, using it will raise a NotImplementedError."); + If it is unavailable, using it will raise a NotImplementedError.\n\ +\n\ +If cloexec is True, set close-on-exec flag."); static PyObject * posix_open(PyObject *self, PyObject *args, PyObject *kwargs) @@ -7008,22 +7052,31 @@ posix_open(PyObject *self, PyObject *arg int flags; int mode = 0777; int dir_fd = DEFAULT_DIR_FD; + int cloexec = Py_DefaultCloexec; int fd; PyObject *return_value = NULL; - static char *keywords[] = {"path", "flags", "mode", "dir_fd", NULL}; + static char *keywords[] = {"path", "flags", "mode", "dir_fd", "cloexec", NULL}; memset(&path, 0, sizeof(path)); path.function_name = "open"; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&i|i$O&:open", keywords, + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O&i|i$O&O&:open", keywords, path_converter, &path, &flags, &mode, #ifdef HAVE_OPENAT - dir_fd_converter, &dir_fd -#else - dir_fd_unavailable, &dir_fd -#endif - )) - return NULL; + dir_fd_converter, &dir_fd, +#else + dir_fd_unavailable, &dir_fd, +#endif + _Py_cloexec_converter, &cloexec)) + return NULL; + + if (cloexec) { +#ifdef MS_WINDOWS + flags |= O_NOINHERIT; +#elif defined(O_CLOEXEC) + flags |= O_CLOEXEC; +#endif + } Py_BEGIN_ALLOW_THREADS #ifdef MS_WINDOWS @@ -7044,6 +7097,23 @@ posix_open(PyObject *self, PyObject *arg goto exit; } +#ifndef MS_WINDOWS + if (cloexec) { +#ifdef O_CLOEXEC + /* Linux kernel older than 2.6.23 ignores O_CLOEXEC flag */ + static int cloexec_works = -1; + int *atomic_flag_works = &cloexec_works; +#else + int *atomic_flag_works = NULL; +#endif + if (_Py_set_cloexec(fd, 1, atomic_flag_works) < 0) { + close(fd); + PyErr_SetFromErrnoWithFilenameObject(PyExc_OSError, path.object); + goto exit; + } + } +#endif + return_value = PyLong_FromLong((long)fd); exit: @@ -7093,39 +7163,93 @@ posix_closerange(PyObject *self, PyObjec PyDoc_STRVAR(posix_dup__doc__, -"dup(fd) -> fd2\n\n\ -Return a duplicate of a file descriptor."); - -static PyObject * -posix_dup(PyObject *self, PyObject *args) -{ + "dup(fd, cloexec=True) -> fd2\n\n" + "Return a duplicate of a file descriptor.\n" + "If cloexec is True, set close-on-exec flag."); + +static PyObject * +posix_dup(PyObject *self, PyObject *args, PyObject *kwargs) +{ + static char *keywords[] = {"fd", "cloexec", NULL}; int fd; - if (!PyArg_ParseTuple(args, "i:dup", &fd)) - return NULL; + int cloexec = Py_DefaultCloexec; + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i|O&:dup", keywords, + &fd, _Py_cloexec_converter, &cloexec)) + return NULL; + if (!_PyVerify_fd(fd)) return posix_error(); - fd = dup(fd); - if (fd < 0) - return posix_error(); +#if defined(HAVE_FCNTL_H) && defined(F_DUPFD_CLOEXEC) + if (cloexec) { + fd = fcntl(fd, F_DUPFD_CLOEXEC, 0); + if (fd < 0) + return posix_error(); + } + else { +#endif + fd = dup(fd); + if (fd < 0) + return posix_error(); + if (cloexec && _Py_set_cloexec(fd, 1, NULL) < 0) + return NULL; +#if defined(HAVE_FCNTL_H) && defined(F_DUPFD_CLOEXEC) + } +#endif return PyLong_FromLong((long)fd); } PyDoc_STRVAR(posix_dup2__doc__, -"dup2(old_fd, new_fd)\n\n\ -Duplicate file descriptor."); - -static PyObject * -posix_dup2(PyObject *self, PyObject *args) -{ +"dup2(fd, fd2, cloexec=True)\n\n\ +Duplicate file descriptor. If cloexec is True, set close-on-exec flag on fd2."); + +static PyObject * +posix_dup2(PyObject *self, PyObject *args, PyObject *kwds) +{ + static char *keywords[] = {"fd", "fd2", "cloexec", NULL}; int fd, fd2, res; - if (!PyArg_ParseTuple(args, "ii:dup2", &fd, &fd2)) + int cloexec = Py_DefaultCloexec; +#ifdef HAVE_DUP3 + /* dup3() is available on Linux 2.6.27+ and glibc 2.9 */ + int dup3_works = -1; +#endif + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "ii|O&:dup2", keywords, + &fd, &fd2, + _Py_cloexec_converter, &cloexec)) return NULL; if (!_PyVerify_fd_dup2(fd, fd2)) return posix_error(); - res = dup2(fd, fd2); - if (res < 0) - return posix_error(); + +#ifdef HAVE_DUP3 + if (dup3_works != 0) { + int flags; + if (cloexec) + flags = O_CLOEXEC; + else + flags = 0; + res = dup3(fd, fd2, flags); + if (res < 0) { + if (dup3_works == -1) + dup3_works = (errno != ENOSYS); + if (dup3_works) + return posix_error(); + } + } + + if (dup3_works == 0) + { +#endif + res = dup2(fd, fd2); + if (res < 0) + return posix_error(); + if (cloexec && _Py_set_cloexec(fd2, 1, NULL) < 0) + return NULL; +#ifdef HAVE_DUP3 + } +#endif + Py_INCREF(Py_None); return Py_None; } @@ -7605,30 +7729,78 @@ posix_isatty(PyObject *self, PyObject *a #ifdef HAVE_PIPE PyDoc_STRVAR(posix_pipe__doc__, -"pipe() -> (read_end, write_end)\n\n\ -Create a pipe."); - -static PyObject * -posix_pipe(PyObject *self, PyObject *noargs) -{ -#if !defined(MS_WINDOWS) +"pipe() -> (read_end, write_end)\n\n" +"Create a pipe. If cloexec is True, set close-on-exec flag."); + +static PyObject * +posix_pipe(PyObject *self, PyObject *args, PyObject *kwargs) +{ + static char *keywords[] = {"cloexec", NULL}; int fds[2]; + int cloexec = Py_DefaultCloexec; +#if defined(MS_WINDOWS) + HANDLE read, write; + SECURITY_ATTRIBUTES attr; + BOOL ok; +#else +#ifdef HAVE_PIPE2 + int flags; +#endif int res; - res = pipe(fds); - if (res != 0) - return posix_error(); - return Py_BuildValue("(ii)", fds[0], fds[1]); -#else /* MS_WINDOWS */ - HANDLE read, write; - int read_fd, write_fd; - BOOL ok; - ok = CreatePipe(&read, &write, NULL, 0); +#endif + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|O&:pipe", keywords, + _Py_cloexec_converter, &cloexec)) + return NULL; + +#if defined(MS_WINDOWS) + attr.nLength = sizeof(attr); + attr.lpSecurityDescriptor = NULL; + attr.bInheritHandle = (cloexec ? FALSE : TRUE); + + Py_BEGIN_ALLOW_THREADS + ok = CreatePipe(&read, &write, &attr, 0); + if (ok) { + fds[0] = _open_osfhandle((Py_intptr_t)read, _O_RDONLY); + ok = (fds[0] != -1); + } + if (ok) { + fds[1] = _open_osfhandle((Py_intptr_t)write, _O_WRONLY); + ok = (fds[1] != -1); + } + Py_END_ALLOW_THREADS + if (!ok) return PyErr_SetFromWindowsErr(0); - read_fd = _open_osfhandle((Py_intptr_t)read, 0); - write_fd = _open_osfhandle((Py_intptr_t)write, 1); - return Py_BuildValue("(ii)", read_fd, write_fd); +#else +#ifdef HAVE_PIPE2 + flags = cloexec ? O_CLOEXEC : 0; + + Py_BEGIN_ALLOW_THREADS + res = pipe2(fds, flags); + Py_END_ALLOW_THREADS + + if (res != 0 && errno == ENOSYS) + { +#endif + Py_BEGIN_ALLOW_THREADS + res = pipe(fds); + Py_END_ALLOW_THREADS + + if (cloexec && res == 0) { + if (_Py_set_cloexec(fds[0], 1, NULL) < 0) + return NULL; + if (_Py_set_cloexec(fds[1], 1, NULL) < 0) + return NULL; + } +#ifdef HAVE_PIPE2 + } +#endif + + if (res != 0) + return PyErr_SetFromErrno(PyExc_OSError); #endif /* MS_WINDOWS */ + return Py_BuildValue("(ii)", fds[0], fds[1]); } #endif /* HAVE_PIPE */ @@ -10181,6 +10353,76 @@ get_terminal_size(PyObject *self, PyObje } #endif /* defined(TERMSIZE_USE_CONIO) || defined(TERMSIZE_USE_IOCTL) */ +#if defined(MS_WINDOWS) \ + || (defined(HAVE_SYS_IOCTL_H) && defined(FIOCLEX) && defined(FIONCLEX)) \ + || defined(HAVE_FCNTL_H) +#define POSIX_SET_CLOEXEC +#endif + +#ifdef POSIX_SET_CLOEXEC +PyDoc_STRVAR(get_cloexec__doc__, + "get_cloexec(fd) -> bool\n" \ + "\n" \ + "Get the close-on-exe flag of the specified file descriptor."); + +static PyObject* +posix_get_cloexec(PyObject *self, PyObject *args) +{ + int fd; + int cloexec; + + if (!PyArg_ParseTuple(args, "i:get_cloexec", &fd)) + return NULL; + +#ifdef MS_WINDOWS + if (!_PyVerify_fd(fd)) { + /* fd may be an handle, ex: a Windows socket */ + fd = _open_osfhandle((Py_intptr_t)fd, _O_RDONLY); + if (fd == -1) { + PyErr_SetFromWindowsErr(0); + return NULL; + } + } +#endif + + cloexec = _Py_get_cloexec(fd); + if (cloexec < 0) + return NULL; + return PyBool_FromLong(cloexec); +} + +PyDoc_STRVAR(set_cloexec__doc__, + "set_cloexec(fd, cloexec=True)\n" \ + "\n" \ + "Set or clear close-on-exe flag on the specified file descriptor."); + +static PyObject* +posix_set_cloexec(PyObject *self, PyObject *args) +{ + int fd; + int cloexec = 1; + + if (!PyArg_ParseTuple(args, "i|i:set_cloexec", &fd, &cloexec)) + return NULL; + +#ifdef MS_WINDOWS + if (!_PyVerify_fd(fd)) { + /* fd may be an handle, ex: a Windows socket */ + fd = _open_osfhandle((Py_intptr_t)fd, _O_RDONLY); + if (fd == -1) { + PyErr_SetFromWindowsErr(0); + return NULL; + } + } +#endif + + if (_Py_set_cloexec(fd, cloexec, NULL) < 0) + return NULL; + Py_RETURN_NONE; +} +#endif /* POSIX_SET_CLOEXEC */ + + static PyMethodDef posix_methods[] = { {"access", (PyCFunction)posix_access, @@ -10345,7 +10587,8 @@ static PyMethodDef posix_methods[] = { #endif #endif /* HAVE_SCHED_H */ #if defined(HAVE_OPENPTY) || defined(HAVE__GETPTY) || defined(HAVE_DEV_PTMX) - {"openpty", posix_openpty, METH_NOARGS, posix_openpty__doc__}, + {"openpty", (PyCFunction)posix_openpty, + METH_VARARGS | METH_KEYWORDS, posix_openpty__doc__}, #endif /* HAVE_OPENPTY || HAVE__GETPTY || HAVE_DEV_PTMX */ #ifdef HAVE_FORKPTY {"forkpty", posix_forkpty, METH_NOARGS, posix_forkpty__doc__}, @@ -10457,8 +10700,10 @@ static PyMethodDef posix_methods[] = { {"close", posix_close, METH_VARARGS, posix_close__doc__}, {"closerange", posix_closerange, METH_VARARGS, posix_closerange__doc__}, {"device_encoding", device_encoding, METH_VARARGS, device_encoding__doc__}, - {"dup", posix_dup, METH_VARARGS, posix_dup__doc__}, - {"dup2", posix_dup2, METH_VARARGS, posix_dup2__doc__}, + {"dup", (PyCFunction)posix_dup, + METH_VARARGS | METH_KEYWORDS, posix_dup__doc__}, + {"dup2", (PyCFunction)posix_dup2, + METH_VARARGS | METH_KEYWORDS, posix_dup2__doc__}, #ifdef HAVE_LOCKF {"lockf", posix_lockf, METH_VARARGS, posix_lockf__doc__}, #endif @@ -10484,7 +10729,8 @@ static PyMethodDef posix_methods[] = { {"fstat", posix_fstat, METH_VARARGS, posix_fstat__doc__}, {"isatty", posix_isatty, METH_VARARGS, posix_isatty__doc__}, #ifdef HAVE_PIPE - {"pipe", posix_pipe, METH_NOARGS, posix_pipe__doc__}, + {"pipe", (PyCFunction)posix_pipe, + METH_VARARGS | METH_KEYWORDS, posix_pipe__doc__}, #endif #ifdef HAVE_PIPE2 {"pipe2", posix_pipe2, METH_O, posix_pipe2__doc__}, @@ -10626,6 +10872,10 @@ static PyMethodDef posix_methods[] = { #if defined(TERMSIZE_USE_CONIO) || defined(TERMSIZE_USE_IOCTL) {"get_terminal_size", get_terminal_size, METH_VARARGS, termsize__doc__}, #endif +#ifdef POSIX_SET_CLOEXEC + {"get_cloexec", posix_get_cloexec, METH_VARARGS, get_cloexec__doc__}, + {"set_cloexec", posix_set_cloexec, METH_VARARGS, set_cloexec__doc__}, +#endif {NULL, NULL} /* Sentinel */ }; diff -r f87d85f7596c -r 9bdfa1a3ea8c Modules/selectmodule.c --- a/Modules/selectmodule.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Modules/selectmodule.c Sat Jan 26 01:09:28 2013 +0100 @@ -911,7 +911,7 @@ newDevPollObject(void) */ limit_result = getrlimit(RLIMIT_NOFILE, &limit); if (limit_result != -1) - fd_devpoll = open("/dev/poll", O_RDWR); + fd_devpoll = _Py_open("/dev/poll", O_RDWR); Py_END_ALLOW_THREADS if (limit_result == -1) { diff -r f87d85f7596c -r 9bdfa1a3ea8c Modules/socketmodule.c --- a/Modules/socketmodule.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Modules/socketmodule.c Sat Jan 26 01:09:28 2013 +0100 @@ -100,13 +100,14 @@ Local naming conventions: /* Socket object documentation */ PyDoc_STRVAR(sock_doc, -"socket([family[, type[, proto]]]) -> socket object\n\ +"socket(family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None, cloexec=None) -> socket object\n\ \n\ Open a socket of the given type. The family argument specifies the\n\ address family; it defaults to AF_INET. The type argument specifies\n\ whether this is a stream (SOCK_STREAM, this is the default)\n\ or datagram (SOCK_DGRAM) socket. The protocol argument defaults to 0,\n\ specifying the default protocol. Keyword arguments are accepted.\n\ +If cloexec is True, set close-on-exec flag.\n\ \n\ A socket object represents one endpoint of a network connection.\n\ \n\ @@ -1641,7 +1642,7 @@ getsockaddrarg(PySocketSockObject *s, Py return 0; } #endif - + #ifdef PF_SYSTEM case PF_SYSTEM: switch (s->sock_proto) { @@ -1649,10 +1650,10 @@ getsockaddrarg(PySocketSockObject *s, Py case SYSPROTO_CONTROL: { struct sockaddr_ctl *addr; - + addr = (struct sockaddr_ctl *)addr_ret; addr->sc_family = AF_SYSTEM; - addr->ss_sysaddr = AF_SYS_CONTROL; + addr->ss_sysaddr = AF_SYS_CONTROL; if (PyUnicode_Check(args)) { struct ctl_info info; @@ -1678,17 +1679,17 @@ getsockaddrarg(PySocketSockObject *s, Py "cannot find kernel control with provided name"); return 0; } - + addr->sc_id = info.ctl_id; addr->sc_unit = 0; } else if (!PyArg_ParseTuple(args, "II", &(addr->sc_id), &(addr->sc_unit))) { PyErr_SetString(PyExc_TypeError, "getsockaddrarg: " "expected str or tuple of two ints"); - + return 0; } - + *len_ret = sizeof(*addr); return 1; } @@ -1805,7 +1806,7 @@ getsockaddrlen(PySocketSockObject *s, so return 1; } #endif - + #ifdef PF_SYSTEM case PF_SYSTEM: switch(s->sock_proto) { @@ -1946,8 +1947,9 @@ get_cmsg_data_len(struct msghdr *msg, st /* s._accept() -> (fd, address) */ static PyObject * -sock_accept(PySocketSockObject *s) +sock_accept(PySocketSockObject *s, PyObject *args, PyObject *kwds) { + static char *kwlist[] = {"cloexec", 0}; sock_addr_t addrbuf; SOCKET_T newfd = INVALID_SOCKET; socklen_t addrlen; @@ -1955,6 +1957,17 @@ sock_accept(PySocketSockObject *s) PyObject *addr = NULL; PyObject *res = NULL; int timeout; + int cloexec = Py_DefaultCloexec; +#if defined(HAVE_ACCEPT4) && defined(SOCK_CLOEXEC) + /* accept4() is available on Linux 2.6.28+ and glibc 2.10 */ + static int accept4_works = -1; +#endif + + /* Get the buffer's memory */ + if (!PyArg_ParseTupleAndKeywords(args, kwds, "|O&:_accept", kwlist, + _Py_cloexec_converter, &cloexec)) + return NULL; + if (!getsockaddrlen(s, &addrlen)) return NULL; memset(&addrbuf, 0, addrlen); @@ -1963,10 +1976,26 @@ sock_accept(PySocketSockObject *s) return select_error(); BEGIN_SELECT_LOOP(s) + Py_BEGIN_ALLOW_THREADS timeout = internal_select_ex(s, 0, interval); if (!timeout) { +#if defined(HAVE_ACCEPT4) && defined(SOCK_CLOEXEC) + if (accept4_works != 0) { + int flags; + if (cloexec) + flags = SOCK_CLOEXEC; + else + flags = 0; + newfd = accept4(s->sock_fd, SAS2SA(&addrbuf), &addrlen, flags); + if (newfd == INVALID_SOCKET && accept4_works == -1) + accept4_works = (errno != ENOSYS); + } + if (accept4_works == 0) + newfd = accept(s->sock_fd, SAS2SA(&addrbuf), &addrlen); +#else newfd = accept(s->sock_fd, SAS2SA(&addrbuf), &addrlen); +#endif } Py_END_ALLOW_THREADS @@ -1979,6 +2008,29 @@ sock_accept(PySocketSockObject *s) if (newfd == INVALID_SOCKET) return s->errorhandler(); + if (cloexec +#if defined(HAVE_ACCEPT4) && defined(SOCK_CLOEXEC) + && accept4_works == 0 +#endif + ) + { + int os_fd; +#ifdef MS_WINDOWS + os_fd = _open_osfhandle((Py_intptr_t)newfd, O_RDWR); + if (os_fd == -1) { + PyErr_SetFromWindowsErr(0); + SOCKETCLOSE(newfd); + goto finally; + } +#else + os_fd = newfd; +#endif + if (_Py_set_cloexec(os_fd, 1, NULL) < 0) { + SOCKETCLOSE(newfd); + goto finally; + } + } + sock = PyLong_FromSocket_t(newfd); if (sock == NULL) { SOCKETCLOSE(newfd); @@ -1999,11 +2051,12 @@ finally: } PyDoc_STRVAR(accept_doc, -"_accept() -> (integer, address info)\n\ +"_accept(cloexec=None) -> (integer, address info)\n\ \n\ Wait for an incoming connection. Return a new socket file descriptor\n\ representing the connection, and the address of the client.\n\ -For IP sockets, the address info is a pair (hostaddr, port)."); +For IP sockets, the address info is a pair (hostaddr, port).\n\ +If cloexec is True, set close-on-exec flag."); /* s.setblocking(flag) method. Argument: False -- non-blocking mode; same as settimeout(0) @@ -3741,8 +3794,8 @@ socket.fromshare()."); /* List of methods for socket objects */ static PyMethodDef sock_methods[] = { - {"_accept", (PyCFunction)sock_accept, METH_NOARGS, - accept_doc}, + {"_accept", (PyCFunction)sock_accept, + METH_VARARGS | METH_KEYWORDS, accept_doc}, {"bind", (PyCFunction)sock_bind, METH_O, bind_doc}, {"close", (PyCFunction)sock_close, METH_NOARGS, @@ -3882,19 +3935,42 @@ sock_new(PyTypeObject *type, PyObject *a /* Initialize a new socket object. */ +#ifdef SOCK_CLOEXEC +/* socket() and socketpair() fail with EINVAL on Linux kernel older + * than 2.6.27 if SOCK_CLOEXEC flag is set in the socket type. */ +static int sock_cloexec_works = -1; +#endif + /*ARGSUSED*/ static int sock_initobj(PyObject *self, PyObject *args, PyObject *kwds) { PySocketSockObject *s = (PySocketSockObject *)self; PyObject *fdobj = NULL; + int cloexec = Py_DefaultCloexec; SOCKET_T fd = INVALID_SOCKET; int family = AF_INET, type = SOCK_STREAM, proto = 0; - static char *keywords[] = {"family", "type", "proto", "fileno", 0}; + static char *keywords[] = {"family", "type", "proto", "fileno", "cloexec", 0}; +#if defined(MS_WINDOWS) + +#ifndef WSA_FLAG_NO_HANDLE_INHERIT +#define WSA_FLAG_NO_HANDLE_INHERIT 0x80 +#endif + + /* WSA_FLAG_NO_HANDLE_INHERIT is supported on Windows 7 with SP1, Windows + * Server 2008 R2 with SP1, and later */ + static int wsa_no_inherit_works = -1; + int *atomic_flag_works = &wsa_no_inherit_works; +#elif defined(SOCK_CLOEXEC) + int *atomic_flag_works = &sock_cloexec_works; +#else + int *atomic_flag_works = NULL; +#endif if (!PyArg_ParseTupleAndKeywords(args, kwds, - "|iiiO:socket", keywords, - &family, &type, &proto, &fdobj)) + "|iiiOO&:socket", keywords, + &family, &type, &proto, &fdobj, + _Py_cloexec_converter, &cloexec)) return -1; if (fdobj != NULL && fdobj != Py_None) { @@ -3902,6 +3978,7 @@ sock_initobj(PyObject *self, PyObject *a /* recreate a socket that was duplicated */ if (PyBytes_Check(fdobj)) { WSAPROTOCOL_INFO info; + DWORD flags = WSA_FLAG_OVERLAPPED; if (PyBytes_GET_SIZE(fdobj) != sizeof(info)) { PyErr_Format(PyExc_ValueError, "socket descriptor string has wrong size, " @@ -3909,9 +3986,11 @@ sock_initobj(PyObject *self, PyObject *a return -1; } memcpy(&info, PyBytes_AS_STRING(fdobj), sizeof(info)); + if (cloexec) + flags |= WSA_FLAG_NO_HANDLE_INHERIT; Py_BEGIN_ALLOW_THREADS fd = WSASocket(FROM_PROTOCOL_INFO, FROM_PROTOCOL_INFO, - FROM_PROTOCOL_INFO, &info, 0, WSA_FLAG_OVERLAPPED); + FROM_PROTOCOL_INFO, &info, 0, flags); Py_END_ALLOW_THREADS if (fd == INVALID_SOCKET) { set_error(); @@ -3936,7 +4015,21 @@ sock_initobj(PyObject *self, PyObject *a } else { Py_BEGIN_ALLOW_THREADS - fd = socket(family, type, proto); +#ifdef SOCK_CLOEXEC + if (cloexec && sock_cloexec_works != 0) { + fd = socket(family, type | SOCK_CLOEXEC, proto); + if (sock_cloexec_works == -1 && fd < 0) { + sock_cloexec_works = (errno != EINVAL); + if (!sock_cloexec_works) { + fd = socket(family, type, proto); + } + } + } + else +#endif + { + fd = socket(family, type, proto); + } Py_END_ALLOW_THREADS if (fd == INVALID_SOCKET) { @@ -3944,6 +4037,21 @@ sock_initobj(PyObject *self, PyObject *a return -1; } } + + if (cloexec && (atomic_flag_works == NULL || *atomic_flag_works != 1)) { + int os_fd; +#ifdef MS_WINDOWS + os_fd = _open_osfhandle((Py_intptr_t)fd, O_RDWR); + if (os_fd == -1) { + PyErr_SetFromWindowsErr(0); + return -1; + } +#else + os_fd = fd; +#endif + if (_Py_set_cloexec(os_fd, 1, atomic_flag_works) < 0) + return -1; + } init_sockobject(s, fd, family, type, proto); return 0; @@ -4534,24 +4642,59 @@ sockets; on some platforms os.dup() won' /*ARGSUSED*/ static PyObject * -socket_socketpair(PyObject *self, PyObject *args) +socket_socketpair(PyObject *self, PyObject *args, PyObject *kwargs) { + static char *kwlist[] = {"family", "type", "proto", "cloexec", 0}; PySocketSockObject *s0 = NULL, *s1 = NULL; SOCKET_T sv[2]; - int family, type = SOCK_STREAM, proto = 0; + int family = AF_INET, type = SOCK_STREAM, proto = 0; PyObject *res = NULL; + int cloexec = Py_DefaultCloexec; +#ifdef SOCK_CLOEXEC + int *atomic_flag_works = &sock_cloexec_works; +#else + int *atomic_flag_works = NULL; +#endif + int ret; #if defined(AF_UNIX) family = AF_UNIX; #else family = AF_INET; #endif - if (!PyArg_ParseTuple(args, "|iii:socketpair", - &family, &type, &proto)) + if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|iiiO&:socketpair", kwlist, + &family, &type, &proto, + _Py_cloexec_converter, &cloexec)) return NULL; + /* Create a pair of socket fds */ - if (socketpair(family, type, proto, sv) < 0) + Py_BEGIN_ALLOW_THREADS +#ifdef SOCK_CLOEXEC + if (cloexec && sock_cloexec_works != 0) { + ret = socketpair(family, type | SOCK_CLOEXEC, proto, sv); + if (sock_cloexec_works == -1 && ret < 0) { + sock_cloexec_works = (errno != EINVAL); + if (!sock_cloexec_works) + ret = socketpair(family, type, proto, sv); + } + } + else +#endif + { + ret = socketpair(family, type, proto, sv); + } + Py_END_ALLOW_THREADS + + if (ret < 0) return set_error(); + + if (cloexec) { + if (_Py_set_cloexec(sv[0], 1, atomic_flag_works) < 0) + goto finally; + if (_Py_set_cloexec(sv[1], 1, atomic_flag_works) < 0) + goto finally; + } + s0 = new_sockobject(sv[0], family, type, proto); if (s0 == NULL) goto finally; @@ -4573,12 +4716,13 @@ finally: } PyDoc_STRVAR(socketpair_doc, -"socketpair([family[, type[, proto]]]) -> (socket object, socket object)\n\ +"socketpair(family=AF_INET, type=SOCK_STREAM, proto=0, cloexec=None) -> (socket object, socket object)\n\ \n\ Create a pair of socket objects from the sockets returned by the platform\n\ socketpair() function.\n\ The arguments are the same as for socket() except the default family is\n\ -AF_UNIX if defined on the platform; otherwise, the default is AF_INET."); +AF_UNIX if defined on the platform; otherwise, the default is AF_INET.\n\ +If cloexec is True, set close-on-exec flag."); #endif /* HAVE_SOCKETPAIR */ @@ -5354,8 +5498,8 @@ static PyMethodDef socket_methods[] = { METH_O, dup_doc}, #endif #ifdef HAVE_SOCKETPAIR - {"socketpair", socket_socketpair, - METH_VARARGS, socketpair_doc}, + {"socketpair", (PyCFunction)socket_socketpair, + METH_VARARGS | METH_KEYWORDS, socketpair_doc}, #endif {"ntohs", socket_ntohs, METH_VARARGS, ntohs_doc}, diff -r f87d85f7596c -r 9bdfa1a3ea8c PC/_msi.c --- a/PC/_msi.c Wed Jan 16 15:19:16 2013 +0100 +++ b/PC/_msi.c Sat Jan 26 01:09:28 2013 +0100 @@ -55,7 +55,7 @@ static FNFCIFREE(cb_free) static FNFCIOPEN(cb_open) { - int result = _open(pszFile, oflag, pmode); + int result = _open(pszFile, oflag | O_NOINHERIT, pmode); if (result == -1) *err = errno; return result; @@ -179,7 +179,7 @@ static FNFCIGETOPENINFO(cb_getopeninfo) CloseHandle(handle); - return _open(pszName, _O_RDONLY | _O_BINARY); + return _open(pszName, _O_RDONLY | _O_BINARY | O_NOINHERIT); } static PyObject* fcicreate(PyObject* obj, PyObject* args) diff -r f87d85f7596c -r 9bdfa1a3ea8c Parser/tokenizer.c --- a/Parser/tokenizer.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Parser/tokenizer.c Sat Jan 26 01:09:28 2013 +0100 @@ -1722,6 +1722,10 @@ PyTokenizer_FindEncodingFilename(int fd, if (fd < 0) { return NULL; } +#ifndef PGEN + _Py_try_set_default_cloexec(fd); +#endif + fp = fdopen(fd, "r"); if (fp == NULL) { return NULL; diff -r f87d85f7596c -r 9bdfa1a3ea8c Python/errors.c --- a/Python/errors.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Python/errors.c Sat Jan 26 01:09:28 2013 +0100 @@ -2,6 +2,7 @@ /* Error handling */ #include "Python.h" +#include #ifndef __STDC__ #ifndef MS_WINDOWS @@ -941,13 +942,17 @@ PyErr_SyntaxLocationEx(const char *filen PyObject * PyErr_ProgramText(const char *filename, int lineno) { + int fd; FILE *fp; int i; char linebuf[1000]; if (filename == NULL || *filename == '\0' || lineno <= 0) return NULL; - fp = fopen(filename, "r" PY_STDIOTEXTMODE); + fd = _Py_open(filename, O_RDONLY); + if (fd < 0) + return NULL; + fp = fdopen(fd, "r" PY_STDIOTEXTMODE); if (fp == NULL) return NULL; for (i = 0; i < lineno; i++) { diff -r f87d85f7596c -r 9bdfa1a3ea8c Python/fileutils.c --- a/Python/fileutils.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Python/fileutils.c Sat Jan 26 01:09:28 2013 +0100 @@ -9,10 +9,20 @@ #include #endif +#if defined(HAVE_SYS_IOCTL_H) +# include +#endif + +#ifdef HAVE_FCNTL_H +#include +#endif /* HAVE_FCNTL_H */ + #ifdef __APPLE__ extern wchar_t* _Py_DecodeUTF8_surrogateescape(const char *s, Py_ssize_t size); #endif +int Py_DefaultCloexec = 0; + PyObject * _Py_device_encoding(int fd) { @@ -547,14 +557,243 @@ int #endif +/* Get the close-on-exec flag of the specified file descriptor. + Return 1 if it is set, 0 if is not set, + raise an exception and return -1 on error. */ +int +_Py_get_cloexec(int fd) +{ +#ifdef MS_WINDOWS + HANDLE handle; + BOOL success; + DWORD flags; + + if (!_PyVerify_fd(fd)) { + handle = INVALID_HANDLE_VALUE; + PyErr_SetFromWindowsErr(ERROR_INVALID_HANDLE); + return -1; + } + + handle = (HANDLE)_get_osfhandle(fd); + if (handle == INVALID_HANDLE_VALUE) { + PyErr_SetFromWindowsErr(0); + return -1; + } + + success = GetHandleInformation(handle, &flags); + if (!success) { + PyErr_SetFromWindowsErr(0); + return -1; + } + + return !(flags & HANDLE_FLAG_INHERIT); +#else + int flags; + + flags = fcntl(fd, F_GETFD, 0); + if (flags == -1) { + PyErr_SetFromErrno(PyExc_OSError); + return -1; + } + return (flags & FD_CLOEXEC); +#endif +} + +static int +set_cloexec(int fd, int cloexec, int raise) +{ +#ifdef MS_WINDOWS + HANDLE handle; + DWORD flags; + BOOL success; +#elif defined(HAVE_SYS_IOCTL_H) && defined(FIOCLEX) && defined(FIONCLEX) + int request; + int err; +#elif defined(HAVE_FCNTL_H) + int flags; + int res; +#endif + +#ifdef MS_WINDOWS + if (!_PyVerify_fd(fd)) { + handle = INVALID_HANDLE_VALUE; + if (raise) + PyErr_SetFromWindowsErr(ERROR_INVALID_HANDLE); + return -1; + } + + handle = (HANDLE)_get_osfhandle(fd); + if (handle == INVALID_HANDLE_VALUE) { + if (raise) + PyErr_SetFromWindowsErr(0); + return -1; + } + + if (cloexec) + flags = 0; + else + flags = HANDLE_FLAG_INHERIT; + success = SetHandleInformation(handle, HANDLE_FLAG_INHERIT, flags); + if (!success) { + if (raise) + PyErr_SetFromWindowsErr(0); + return -1; + } + return 0; +#elif defined(HAVE_SYS_IOCTL_H) && defined(FIOCLEX) && defined(FIONCLEX) + if (cloexec) + request = FIOCLEX; + else + request = FIONCLEX; + err = ioctl(fd, request); + if (err) { + if (raise) + PyErr_SetFromErrno(PyExc_OSError); + return -1; + } + return 0; +#elif defined(HAVE_FCNTL_H) + flags = fcntl(fd, F_GETFD); + if (flags < 0) { + if (raise) + PyErr_SetFromErrno(PyExc_OSError); + return -1; + } + + if (cloexec) + flags |= FD_CLOEXEC; + else + flags &= ~FD_CLOEXEC; + res = fcntl(fd, F_SETFD, flags); + if (res < 0) { + if (raise) + PyErr_SetFromErrno(PyExc_OSError); + return -1; + } + return 0; +#else + if (raise) + PyErr_SetString(PyExc_NotImplementedError, + "close-on-exec flag is not supported on your platform"); + return -1; +#endif +} + +/* Set or clear close-on-exec flag of the specified file descriptor. + On success: return 0, on error: raise an exception if raise is nonzero + and return -1. */ +int +_Py_set_cloexec(int fd, int cloexec, int *atomic_flags_works) +{ + if (atomic_flags_works != NULL) { + if (*atomic_flags_works == -1) { + int works = _Py_get_cloexec(fd); + if (works == -1) + return -1; + *atomic_flags_works = works; + } + + if (*atomic_flags_works) + return 0; + } + + return set_cloexec(fd, cloexec, 1); +} + +/* Try to set close-on-exec flag of the specified file descriptor. + If setting the close-on-exec flag failed, ignore the error. */ +void +_Py_try_set_cloexec(int fd, int cloexec) +{ + (void)set_cloexec(fd, cloexec, 0); +} + +/* Try to set the default value of the close-on-exec flag of the specified file + descriptor. If setting or clearing the close-on-exec flag failed, ignore the + error. + + The function considers that the close-on-exec flag is not set on the + specified file descriptor. */ +void +_Py_try_set_default_cloexec(int fd) +{ + if (!Py_DefaultCloexec) { + /* close-on-exec flag is cleared by default */ + return; + } + (void)set_cloexec(fd, 1, 0); +} + +static int +open_cloexec(const char *pathname, int flags, int cloexec) +{ + int fd; +#ifdef O_CLOEXEC + static int cloexec_works = -1; +#endif + + if (!cloexec) + return open(pathname, flags); + +#ifdef MS_WINDOWS + flags |= O_NOINHERIT; +#elif defined(O_CLOEXEC) + flags |= O_CLOEXEC; +#endif + fd = open(pathname, flags); + if (fd < 0) + return fd; + +#if defined(O_CLOEXEC) && !defined(MS_WINDOWS) + if (cloexec_works == -1) { + int flags = fcntl(fd, F_GETFD, 0); + if (flags != -1) + cloexec_works = (flags & FD_CLOEXEC); + else + cloexec_works = 0; + } + + if (!cloexec_works) { + /* Linux kernel older than 2.6.23 ignores O_CLOEXEC flag */ + (void)set_cloexec(fd, 1, 0); + } +#elif !defined(MS_WINDOWS) + (void)set_cloexec(fd, 1, 0); +#endif + return fd; +} + +/* Open a file with the specified flags (wrapper to open() function). + + Try to apply the default value of the close-on-exec flag on the newly + created file descriptor. If setting the close-on-exec flag failed, ignore + the error. */ +int +_Py_open(const char *pathname, int flags) +{ + return open_cloexec(pathname, flags, Py_DefaultCloexec); +} + +/* Open a file with the specified flags (wrapper to open() function). + + Try to set close-on-exec flag of the newly created file descriptor. + If setting the close-on-exec flag failed, ignore the error. */ +int +_Py_open_cloexec(const char *pathname, int flags) +{ + return open_cloexec(pathname, flags, 1); +} + /* Open a file. Use _wfopen() on Windows, encode the path to the locale - encoding and use fopen() otherwise. */ + encoding and use fopen() otherwise. + Try to set close-on-exec flag of the newly created file descriptor. + If setting the close-on-exec flag failed, ignore the error. */ FILE * _Py_wfopen(const wchar_t *path, const wchar_t *mode) { + FILE *f; #ifndef MS_WINDOWS - FILE *f; char *cpath; char cmode[10]; size_t r; @@ -570,19 +809,27 @@ FILE * PyMem_Free(cpath); return f; #else - return _wfopen(path, mode); + f = _wfopen(path, mode); #endif + if (f == NULL) + return NULL; + if (Py_DefaultCloexec) + (void)set_cloexec(fileno(f), 1, 0); + return f; } -/* Call _wfopen() on Windows, or encode the path to the filesystem encoding and - call fopen() otherwise. +/* Open a file. Call _wfopen() on Windows, or encode the path to the filesystem + encoding and call fopen() otherwise. + + Try to set close-on-exec flag of the newly created file descriptor. + If setting the close-on-exec flag failed, ignore the error. Return the new file object on success, or NULL if the file cannot be open or - (if PyErr_Occurred()) on unicode error */ - + (if PyErr_Occurred()) on unicode error. */ FILE* _Py_fopen(PyObject *path, const char *mode) { + FILE *f; #ifdef MS_WINDOWS wchar_t *wpath; wchar_t wmode[10]; @@ -602,16 +849,19 @@ FILE* if (usize == 0) return NULL; - return _wfopen(wpath, wmode); + f = _wfopen(wpath, wmode); #else - FILE *f; PyObject *bytes; if (!PyUnicode_FSConverter(path, &bytes)) return NULL; f = fopen(PyBytes_AS_STRING(bytes), mode); Py_DECREF(bytes); +#endif + if (f == NULL) + return NULL; + if (Py_DefaultCloexec) + (void)set_cloexec(fileno(f), 1, 0); return f; -#endif } #ifdef HAVE_READLINK @@ -728,3 +978,26 @@ wchar_t* #endif } +/* Converter for PyArg_ParseTuple() to parse the 'cloexec' parameter: + + - if the argument is None, return sys.getdefaultcloexec() + - otherwise, return bool(argument) + + Return 0 on error, 1 on success. +*/ +int +_Py_cloexec_converter(PyObject* arg, void* addr) +{ + int cloexec; + if (arg == Py_None) { + cloexec = Py_DefaultCloexec; + } + else { + cloexec = PyObject_IsTrue(arg); + if (cloexec == -1) + return 0; + } + *(int *)addr = cloexec; + return 1; +} + diff -r f87d85f7596c -r 9bdfa1a3ea8c Python/import.c --- a/Python/import.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Python/import.c Sat Jan 26 01:09:28 2013 +0100 @@ -1807,6 +1807,7 @@ imp_load_dynamic(PyObject *self, PyObjec PyErr_SetFromErrno(PyExc_IOError); return NULL; } + _Py_try_set_cloexec(fileno(fp), 1); } else fp = NULL; diff -r f87d85f7596c -r 9bdfa1a3ea8c Python/pythonrun.c --- a/Python/pythonrun.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Python/pythonrun.c Sat Jan 26 01:09:28 2013 +0100 @@ -29,6 +29,10 @@ #include #endif +#ifdef HAVE_FCNTL_H +#include +#endif /* HAVE_FCNTL_H */ + #ifdef MS_WINDOWS #undef BYTE #include "windows.h" @@ -275,6 +279,8 @@ void check its value further. */ if ((p = Py_GETENV("PYTHONHASHSEED")) && *p != '\0') Py_HashRandomizationFlag = add_flag(Py_HashRandomizationFlag, p); + if ((p = Py_GETENV("PYTHONCLOEXEC")) && *p != '\0') + Py_DefaultCloexec = 1; _PyRandom_Init(); @@ -1418,11 +1424,17 @@ PyRun_SimpleFileExFlags(FILE *fp, const len = strlen(filename); ext = filename + len - (len > 4 ? 4 : 0); if (maybe_pyc_file(fp, filename, ext, closeit)) { + int fd; FILE *pyc_fp; /* Try to run a pyc file. First, re-open in binary */ if (closeit) fclose(fp); - if ((pyc_fp = fopen(filename, "rb")) == NULL) { + fd = _Py_open(filename, O_RDONLY); + if (fd != -1) + pyc_fp = fdopen(fd, "rb"); + else + pyc_fp = NULL; + if (pyc_fp == NULL) { fprintf(stderr, "python: Can't reopen .pyc file\n"); goto done; } diff -r f87d85f7596c -r 9bdfa1a3ea8c Python/random.c --- a/Python/random.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Python/random.c Sat Jan 26 01:09:28 2013 +0100 @@ -128,7 +128,7 @@ dev_urandom_noraise(char *buffer, Py_ssi assert (0 < size); - fd = open("/dev/urandom", O_RDONLY); + fd = _Py_open("/dev/urandom", O_RDONLY); if (fd < 0) Py_FatalError("Failed to open /dev/urandom"); @@ -160,11 +160,8 @@ dev_urandom_python(char *buffer, Py_ssiz if (size <= 0) return 0; - Py_BEGIN_ALLOW_THREADS - fd = open("/dev/urandom", O_RDONLY); - Py_END_ALLOW_THREADS - if (fd < 0) - { + fd = _Py_open("/dev/urandom", O_RDONLY); + if (fd < 0) { PyErr_SetString(PyExc_NotImplementedError, "/dev/urandom (or equivalent) not found"); return -1; diff -r f87d85f7596c -r 9bdfa1a3ea8c Python/sysmodule.c --- a/Python/sysmodule.c Wed Jan 16 15:19:16 2013 +0100 +++ b/Python/sysmodule.c Sat Jan 26 01:09:28 2013 +0100 @@ -1057,6 +1057,30 @@ PyDoc_STRVAR(sys_clear_type_cache__doc__ Clear the internal type lookup cache."); +static PyObject * +sys_getdefaultcloexec(PyObject* self) +{ + return PyBool_FromLong(Py_DefaultCloexec); +} + +PyDoc_STRVAR(getdefaultcloexec_doc, +"getdefaultcloexec() -> bool\n\ +\n\ +Return the current default value of 'cloexec' (close-on-exec) parameter."); + +static PyObject * +sys_setdefaultcloexec(PyObject* self) +{ + Py_DefaultCloexec = 1; + Py_RETURN_NONE; +} + +PyDoc_STRVAR(setdefaultcloexec_doc, +"setdefaultcloexec()\n\ +\n\ +Set the default value of the 'cloexec' (close-on-exec) parameter to True."); + + static PyMethodDef sys_methods[] = { /* Might as well keep this in alphabetic order */ {"callstats", (PyCFunction)PyEval_GetCallStats, METH_NOARGS, @@ -1131,6 +1155,10 @@ static PyMethodDef sys_methods[] = { {"call_tracing", sys_call_tracing, METH_VARARGS, call_tracing_doc}, {"_debugmallocstats", sys_debugmallocstats, METH_VARARGS, debugmallocstats_doc}, + {"getdefaultcloexec", (PyCFunction)sys_getdefaultcloexec, + METH_NOARGS, getdefaultcloexec_doc}, + {"setdefaultcloexec", (PyCFunction)sys_setdefaultcloexec, + METH_NOARGS, setdefaultcloexec_doc}, {NULL, NULL} /* sentinel */ }; diff -r f87d85f7596c -r 9bdfa1a3ea8c configure.ac --- a/configure.ac Wed Jan 16 15:19:16 2013 +0100 +++ b/configure.ac Sat Jan 26 01:09:28 2013 +0100 @@ -2793,7 +2793,7 @@ AC_CHECK_FUNCS(alarm accept4 setitimer g sigtimedwait sigwait sigwaitinfo snprintf strftime strlcpy symlinkat sync \ sysconf tcgetpgrp tcsetpgrp tempnam timegm times tmpfile tmpnam tmpnam_r \ truncate uname unlinkat unsetenv utimensat utimes waitid waitpid wait3 wait4 \ - wcscoll wcsftime wcsxfrm writev _getpty) + wcscoll wcsftime wcsxfrm writev _getpty dup3) AC_CHECK_DECL(dirfd, AC_DEFINE(HAVE_DIRFD, 1, diff -r f87d85f7596c -r 9bdfa1a3ea8c pyconfig.h.in --- a/pyconfig.h.in Wed Jan 16 15:19:16 2013 +0100 +++ b/pyconfig.h.in Sat Jan 26 01:09:28 2013 +0100 @@ -193,6 +193,9 @@ /* Define to 1 if you have the `dup2' function. */ #undef HAVE_DUP2 +/* Define to 1 if you have the `dup3' function. */ +#undef HAVE_DUP3 + /* Defined when any dynamic module loading is enabled. */ #undef HAVE_DYNAMIC_LOADING