classification
Title: doc: subprocess: inheritance of std descriptors inconsistent
Type: behavior Stage: patch review
Components: Documentation, Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: astrand, docs@python, eryksun, iritkatriel, kiilerix, ndparker, rosslagerwall
Priority: normal Keywords: easy, patch

Created on 2005-06-26 13:37 by ndparker, last changed 2021-04-29 11:14 by vstinner.

Files
File name Uploaded Description Edit
1227748.patch rosslagerwall, 2011-01-15 06:47 Doc patch
Messages (6)
msg25643 - (view) Author: André Malo (ndparker) * Date: 2005-06-26 13:37
The inheritance of std descriptors is inconsistent
between Unix and Windows implementations.

If one calls Popen with stdin = stdout = stderr = None,
the caller's std descriptors are inherited on *x, but
not on Windows, because of the following optimization
(from subprocess.py r1.20):

   655	        def _get_handles(self, stdin, stdout,
stderr):
   656	            """Construct and return tupel with
IO objects:
   657	            p2cread, p2cwrite, c2pread,
c2pwrite, errread, errwrite
   658	            """
   659	            if stdin is None and stdout is None
and stderr is None:
   660	                return (None, None, None, None,
None, None)
   661	

I suggest to just remove those lines 659 and 660. The
current workaround is to duplicate the handles by the
application and supply an own STARTUPINFO structure.
msg25644 - (view) Author: Peter Ă…strand (astrand) * (Python committer) Date: 2007-01-29 20:54
>If one calls Popen with stdin = stdout = stderr = None,
>the caller's std descriptors are inherited on *x, but
>not on Windows, 

This is a correct observation. However, the current implementation is not necessarily wrong. This could instead be seen as a consequence of the different environments. The subprocess documentation states that "With None, no redirection will occur". So, it becomes an interpretation of what this really mean. Since the "default" behaviour on UNIX is to inherit and the default behaviour on Windows is to attach the standard handles to (an often newly created) console window, one could argue that this fits fairly good with the description "no redirection will occur". 

If we would change this, so that the parents handles are always inherited, then how would you specify that you want to attach the standard handles to the new console window? 

For best flexibility, the API should allow both cases: Both inherit all handles from the parent as well as attaching all standard handles to the new console window. As you point out, the current API allows this. So why change this?

One thing that's clearly an bug is the second part of the documentation:

"With None, no redirection will occur; the child's file handles will be inherited from the
parent"

This is currently only true on UNIX. If we should keep the current behaviour, at least the comment needs to be fixed. 
msg63330 - (view) Author: Mads Kiilerich (kiilerix) * Date: 2008-03-06 18:48
Note to others searching for a solution to this and similar problems: 
http://svn.python.org/view/python/trunk/Lib/subprocess.py?rev=60115&view=auto
shows that this now (for 2.6?) has been changed so that close_fds now
controls inheritance through the CreateProcess parameter bInheritHandles

Thanks!
msg126315 - (view) Author: Ross Lagerwall (rosslagerwall) (Python committer) Date: 2011-01-15 06:47
I think all that is needed is a documentation patch. Attached is a doc patch which changes the doc to explicitly describe what happens on unix & windows (as described by Peter).
msg392244 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-04-28 18:36
The patch needs to be converted into a github PR.
msg392270 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-04-29 00:29
> If one calls Popen with stdin = stdout = stderr = None,
> the caller's std descriptors are inherited on *x, but
> not on Windows

In the default case, CreateProcessW() is called with bInheritHandles as false and without explicitly setting the standard-handle values. Also, creationflags is 0 in the default case, not CREATE_NEW_CONSOLE, CREATE_NO_WINDOW, or DETACHED_PROCESS. In this case, the child's standard handles are implicitly duplicated from the parent if the child shares the parent's console session. Contra "1227748.patch", the standard handles are not "attached to the console window". For example, if the parent's stdout is redirected to a handle for an open disk file, the child's stdout will be redirected to the same open file. It has nothing to do with a window, and the console session doesn't even need to have a window (i.e. created with CREATE_NO_WINDOW; or a pseudoconsole session).

In particular, if the executable image of the child is a console application, the system duplicates the parent's standard handles to the child, as well as a handle for the parent's console session, if any. If the child can connect to the parent's console at startup, then the duplicated standard handles are used, whatever they are. (In general, they need to be handles for pipe, character, or disk files opened in synchronous mode, as required by C standard I/O.) If the child can't connect to a console session, it allocates a new console and opens the console "Input" and "Output" as its standard handles. 

On the other hand, if the executable image of the child is not a console application, the system does not implicitly duplicate standard handles from the parent to the child. The window manager and graphical shell even explicitly reuse the standard-handle values (via STARTUPINFO) to implement features that aren't I/O file related when running GUI apps, including configuring an activation hotkey (stdin) or passing a handle for the preferred output monitor (stdout).
History
Date User Action Args
2021-04-29 11:14:57vstinnersetnosy: - vstinner
2021-04-29 00:29:32eryksunsetnosy: + eryksun
messages: + msg392270
2021-04-28 18:36:13iritkatrielsetnosy: + iritkatriel
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.1, Python 2.7, Python 3.2
messages: + msg392244

keywords: + easy
title: subprocess: inheritance of std descriptors inconsistent -> doc: subprocess: inheritance of std descriptors inconsistent
2013-08-13 22:19:35vstinnersetnosy: + vstinner
2011-02-05 09:50:00ned.deilysetassignee: astrand -> docs@python

nosy: + docs@python
stage: needs patch -> patch review
2011-01-15 06:47:51rosslagerwallsetfiles: + 1227748.patch

nosy: + rosslagerwall
messages: + msg126315

components: + Documentation
keywords: + patch
2010-08-21 16:06:18BreamoreBoysetstage: needs patch
type: behavior
versions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.4
2008-03-06 18:48:18kiilerixsetnosy: + kiilerix
messages: + msg63330
2005-06-26 13:37:33ndparkercreate