Author milko.krachounov
Recipients Giovanni.Bajo, gregory.p.smith, milko.krachounov, paul.moore
Date 2010-12-10.18:46:59
SpamBayes Score 2.213e-10
Marked as misclassified No
Message-id <1292006821.43.0.485126143882.issue7213@psf.upfronthosting.co.za>
In-reply-to
Content
I'd offer two ideas.

1. Add a constant DISREGARD_FDS to the subprocess module could help. It would allow the user to specify his intent, and let the implementation choose the best action. Popen(..., close_fds=subprocess.DISREGARD_FDS) would mean that any fds additional fds are not needed, and can be closed if it is determined that it is the best course of action.

So, if DISREGARD_FDS gets passed on Windows assume close_fds=False, while on POSIX assume close_fds=True (although this has its downsides, see at the bottom).
2. Set the CLOEXEC flag on the other side of the pipes after the fork. With the attached patch, I don't get the issue I reported:

Python 3.2b1 (py3k:87158, Dec 10 2010, 20:13:57) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from subprocess import *
>>> p1 = Popen(['cat'], stdin=PIPE, stdout=PIPE, close_fds=False)
>>> p2 = Popen(['grep', 'a'], stdin=p1.stdout, stdout=PIPE, close_fds=False)
>>> p1.stdin.write(b"aaaaaaaaaaaaaaaa\n")
17
>>> p1.stdin.close()
>>> p2.stdout.read()
b'aaaaaaaaaaaaaaaa\n'
>>> 

Additional problem with close_fds=True is that if MAXFD is huge, it can cause a huge problem with performance, so perhaps the default being True isn't all that great. On Linux you can close only the open fds, but I'm not sure if that's possible on other POSIX systems.
History
Date User Action Args
2010-12-10 18:47:01milko.krachounovsetrecipients: + milko.krachounov, gregory.p.smith, paul.moore, Giovanni.Bajo
2010-12-10 18:47:01milko.krachounovsetmessageid: <1292006821.43.0.485126143882.issue7213@psf.upfronthosting.co.za>
2010-12-10 18:47:00milko.krachounovlinkissue7213 messages
2010-12-10 18:47:00milko.krachounovcreate