classification
Title: pipe fd handling issues in subprocess.py on POSIX
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: anissen, astrand, djmitche, gregory.p.smith, jfinkels, pitrou, pw, terry.reedy
Priority: normal Keywords: patch

Created on 2007-09-21 18:42 by anissen, last changed 2011-01-03 21:16 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
fix_fileno.py anissen, 2007-09-21 18:42
subprocess.fix_fileno.diff anissen, 2007-09-21 18:44
1187-dustin.patch djmitche, 2008-05-16 03:49 review
issue1187.patch jfinkels, 2010-09-16 17:49 Adapted 1187-dustin.patch for py3k branch.
Messages (10)
msg56079 - (view) Author: Andrew Nissen (anissen) Date: 2007-09-21 18:42
Revision 53293 appears to have missed some of the logic inherent in the
previous code.  There also appears to be problems with the way that the
dup2 calls are made that can result in a behavior different then
intended under a number of circumstances.

fix_fileno.py demonstrates the improper behavior
msg56080 - (view) Author: Andrew Nissen (anissen) Date: 2007-09-21 18:44
This patch (subprocess.fix_fileno.udiff) made against subprocess.py
(Revision 55604)appears to give the desired behavior
msg66577 - (view) Author: Dustin J. Mitchell (djmitche) * Date: 2008-05-10 19:58
I see that, on running your fix_fileno.py, nothing is output to 
/tmp/stdout.test.  I don't necessarily see the link to your fix.  Could 
you describe the problem and/or add comments to your patch to explain why 
these checks are made?
msg66807 - (view) Author: Andrew Nissen (anissen) Date: 2008-05-13 23:09
In reference to Dustin's entry: That's the point; the expected behavior
is that subprocess should write data to the named files, without the
fix, it doesn't.  With the subprocess module as it stands, there are a
number of cases that will not behave as the user expects.  There were
two issues that were being addressed:

1) If a user passes in a file descriptor that is in the range 0-2, the
dup2 calls end up closing the file being passed in.

2) The other issue is the close code could end up closing file
descriptors it really shouldn't.  For example if p2cread == 1, the code
ends up closing the fd even though it really probably shouldn't.  On a
side note, I should have used a list for dup_fds instead of a dictionary.

It's been a while and I've not spent a huge amount of time getting back
up to speed; if there are any questions let me know and I'll spend some
more time on this.
msg66891 - (view) Author: Dustin J. Mitchell (djmitche) * Date: 2008-05-16 03:49
Hmm.. I see why you didn't write a unit test for this!

Attached is a patch with a unit test that tickles this behavior, along 
with Andrew's fix.
msg70639 - (view) Author: Dustin J. Mitchell (djmitche) * Date: 2008-08-02 19:50
Thoughts on this patch?
msg78205 - (view) Author: (pw) Date: 2008-12-22 18:24
Stumbled on this bug a different way in 2.5.2.  My code
worked in 2.5, which included change 51793, but fails in 2.5.1
and beyond due to the reimplementation in 53294.  Dustin's
patch, applied by hand to 2.5.2, fixes things for me.
msg112755 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-08-04 04:37
I am assuming 3.x has problem. Would be good to verify.

1187-dustin.patch has test + fix.
msg116582 - (view) Author: Jeffrey Finkelstein (jfinkels) * Date: 2010-09-16 17:49
Here's a patch which adds the test from 1187-dustin.patch for the py3k branch. The test passes without any additional changes to the py3k code.
msg125243 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-03 21:16
Additional test modified and committed in r87712, thank you!
History
Date User Action Args
2011-01-03 21:16:18pitrousetstatus: open -> closed

versions: - Python 3.1, Python 2.7
nosy: + pitrou

messages: + msg125243
resolution: fixed
stage: patch review -> resolved
2010-09-16 17:49:03jfinkelssetfiles: + issue1187.patch
nosy: + jfinkels
messages: + msg116582

2010-08-04 04:37:12terry.reedysetversions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6, Python 2.5
nosy: + terry.reedy

messages: + msg112755

stage: patch review
2010-03-04 03:09:42gregory.p.smithsetassignee: astrand -> gregory.p.smith

nosy: + gregory.p.smith
2008-12-22 18:24:19pwsetnosy: + pw
messages: + msg78205
versions: + Python 2.5
2008-08-02 19:50:28djmitchesetmessages: + msg70639
2008-05-16 03:49:14djmitchesetfiles: + 1187-dustin.patch
messages: + msg66891
2008-05-13 23:09:13anissensetmessages: + msg66807
2008-05-10 19:58:16djmitchesetnosy: + djmitche
messages: + msg66577
2007-09-23 04:08:43jafosetpriority: normal
assignee: astrand
nosy: + astrand
versions: + Python 2.6, - Python 2.4
2007-09-22 22:25:28loewissetkeywords: + patch
2007-09-21 18:44:00anissensetfiles: + subprocess.fix_fileno.diff
messages: + msg56080
2007-09-21 18:42:09anissencreate