classification
Title: subprocess.py leaks fd in communicate
Type: resource usage Stage:
Components: Library (Lib) Versions: Python 2.6, Python 2.5
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: gregory.p.smith, jrosdahl, zanella
Priority: low Keywords: patch

Created on 2008-05-08 12:22 by jrosdahl, last changed 2008-06-01 23:45 by gregory.p.smith. This issue is now closed.

Files
File name Uploaded Description Edit
subprocess-fd-problem.py jrosdahl, 2008-05-08 12:22 Code triggering the problem
Messages (4)
msg66415 - (view) Author: Joel Rosdahl (jrosdahl) * Date: 2008-05-08 12:22
The optimization in SVN rev 38556 seems to have changed
Popen.communicate's behavior when stdout is subprocess.PIPE (and maybe
for other cases as well).

See the attached file. In Python 2.4.5, all three counts are the same.
In Python 2.5.2, the middle count has increased by 1. In other words: A
file descriptor is leaked until the last reference to the Popen instance
is dropped.
msg67068 - (view) Author: Rafael Zanella (zanella) Date: 2008-05-19 18:08
I don't know a lot about the matter at hand, that's why I'm not gonna
append a patch.

On "_communicate()" after a pipe is read it's closed, doing the same on
"communicate()" seems to solve the issue of the extra pipe:

"""
 if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
            stdout = None
            stderr = None
            if self.stdin:
                if input:
                    self.stdin.write(input)
                self.stdin.close()
            elif self.stdout:
                stdout = self.stdout.read()
+             self.stdout.close()
            elif self.stderr:
                stderr = self.stderr.read()
+             self.stderr.close()
            self.wait()
            return (stdout, stderr)

"""

Tested on "Python 2.6a2+ (trunk:62767M, May 19 2008, 13:11:07)".
msg67391 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2008-05-26 20:24
Thanks zanella.  Fixed in 2.6 trunk r63724.  I created a unit test based
on jrosdahl's example code.

I will backport it to release25-maint after it has baked in trunk for a bit.
msg67618 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2008-06-01 23:45
fixed in release25-maint r63881.
History
Date User Action Args
2008-06-01 23:45:30gregory.p.smithsetstatus: open -> closed
resolution: accepted
messages: + msg67618
versions: + Python 2.5
2008-05-26 20:24:30gregory.p.smithsetpriority: low
messages: + msg67391
versions: - Python 2.5
2008-05-26 19:57:05gregory.p.smithsetkeywords: + patch
assignee: gregory.p.smith
nosy: + gregory.p.smith
versions: + Python 2.6
2008-05-19 18:08:25zanellasetnosy: + zanella
messages: + msg67068
2008-05-08 12:22:43jrosdahlcreate