This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: subprocess.Popen fails with stdout=PIPE, stderr=PIPE if standard descriptors (0, 1, 2) are closed.
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Subprocess error if fds 0,1,2 are closed
View: 10806
Assigned To: Nosy List: Thomas.Claveirole, amaury.forgeotdarc, gregory.p.smith, pitrou, rosslagerwall
Priority: normal Keywords: patch

Created on 2010-09-20 17:59 by Thomas.Claveirole, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
i9905_v1.patch rosslagerwall, 2011-01-02 14:02 Patch + test for issue
Messages (4)
msg116957 - (view) Author: Thomas Claveirole (Thomas.Claveirole) Date: 2010-09-20 17:59
Hello,

Here is a code that exhibits an invalid behavior (Python 2.6.6):
---8<---
import subprocess, os

os.close(0) # Works correctly if any of these two are commented out.
os.close(2)

print subprocess.Popen('echo foo>&2', shell=True,
                       stdout=subprocess.PIPE,
                       stderr=subprocess.PIPE).communicate()
--->8---

When run, the output is:
('', '')

While it should be:
('', 'foo\n')

When analyzing the code with strace the problem gets clearer:
$ strace -f -e pipe,fork,dup2,close ./Popen-bug.py
[...]
5085  pipe([0, 2])                      = 0  # Creates the pipes.
5085  pipe([3, 4])                      = 0
5085  pipe([5, 6])                      = 0
[...] # In this skipped part Popen() closes useless pipe endpoints.
5086  dup2(2, 1)                        = 1 # stdout setup.
5086  dup2(4, 2)                        = 2 # stderr setup.
5086  close(2)                          = 0
[...]

The last "close(2)" is the error: apparently Popen() tries to close the remaining pipe endpoints (as should theoretically be done) but fails to see that the endpoint created by pipe([0, 2]) has already been closed during the previous dup2(4, 2) and that the file descriptor 2 is now the standard error.  Therefore, Popen incorrectly closes the standard error.

To fix that, Popen should check, before closing the remaining pipe endpoints, that these endpoints are not the one that just get closed by the two previous dup2s.

Best regards,
msg117242 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-09-23 22:20
issue6610 looks very similar. issue9074 may also be related.
msg125050 - (view) Author: Ross Lagerwall (rosslagerwall) (Python committer) Date: 2011-01-02 14:02
OK here is a patch + tests. Basically, it makes sure that the fd that it is closing is not 0, 1 or 2.

I've set it for 2.7, 3.1 and 3.2.
msg125212 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-03 18:47
Fixed in issue10806.
History
Date User Action Args
2022-04-11 14:57:06adminsetgithub: 54114
2011-01-03 18:47:38pitrousetstatus: open -> closed

nosy: + pitrou
messages: + msg125212

superseder: Subprocess error if fds 0,1,2 are closed
resolution: duplicate
2011-01-02 14:50:54georg.brandllinkissue9074 superseder
2011-01-02 14:02:35rosslagerwallsetfiles: + i9905_v1.patch
versions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6
nosy: + rosslagerwall

messages: + msg125050

keywords: + patch
2010-09-23 22:20:44amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg117242
2010-09-23 21:03:43r.david.murraysetnosy: + gregory.p.smith
2010-09-20 17:59:33Thomas.Claveirolecreate