This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author richardxia
Recipients eryksun, gregory.p.smith, izbyshev, richardxia, vstinner
Date 2021-10-27.00:45:37
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1635295537.66.0.356791473776.issue42738@roundup.psfhosted.org>
In-reply-to
Content
I'd like to provide another, non-performance-related use case for changing the default value of Popen's close_fds parameters back to False.

In some scenarios, a (non-Python) parent process may want its descendant processes to inherit a particular file descriptor and for each descendant process to pass on that file descriptor its own children. In this scenario, a Python program may just be an intermediate script that calls out to multiple subprocesses, and closing the inheritable file descriptors by default would interfere with the parent process's ability to pass on that file descriptor to descendants.

As a concrete example, we have a (non-Python) build system and task runner that orchestrates many tasks to run in parallel. Some of those tasks end up invoking Python scripts that use subprocess.run() to run other programs. Our task runner intentionally passes an inheritable file descriptor that is unique to each task as a form of a keep-alive token; if the child processes continue to pass inheritable file descriptors to their children, then we can determine whether all of the processes spawned from a task have terminated by checking whither the last open handle to that file descriptor has been closed. This is particularly important when a processes exits before its children, sometimes uncleanly due to being force killed by the system or by a user.

In our use case, Python's default value of close_fds=True interferes with our tracking scheme, since it prevents Python's subprocesses from inheriting that file descriptor, even though that file descriptor has intentionally been made inheritable.

While we are able to work around the issue by explicitly setting close_fds=False in as much of our Python code as possible, it's difficult to enforce this globally since we have many small Python scripts. We also have no control over any third party libraries that may possibly call Popen.

Regarding security, PEP 446 already makes it so that any files opened from within a Python program are non-inheritable by default, which I agree is a good default. One can make the argument that it's not Python's job to enforce a security policy on file descriptors that a Python process has inherited from a parent process, since Python cannot distinguish from descriptors that were accidentally or intentionally inherited.
History
Date User Action Args
2021-10-27 00:45:37richardxiasetrecipients: + richardxia, gregory.p.smith, vstinner, eryksun, izbyshev
2021-10-27 00:45:37richardxiasetmessageid: <1635295537.66.0.356791473776.issue42738@roundup.psfhosted.org>
2021-10-27 00:45:37richardxialinkissue42738 messages
2021-10-27 00:45:37richardxiacreate