If we start a short-lived process which finishes before we begin communicating with it (e.g. by crashing), we can receive a SIGPIPE due to the receiving process no longer existing.  This becomes an EPIPE, which becomes an:
  OSError: [Errno 32] Broken pipe

Arguably this is a bug; if the subprocess could crash, the user currently has to check for it by both monitoring the returncode _and_ catching OSError (then examining for this specific errno), which seems ugly to me.

I'm attaching a patch for subprocess which handles this case, masking the OSError within the Popen implementation, so that you have to test for it within the returncode, in one place, rather than wrap every call with a try/except.

It could be argued that this is incorrect, as it masks under-reads of stdin by the subprocess.  However I believe a sanely-written subprocess ought to indicate success/failure back with its return code.

This was originally filed downstream within the Red Hat bugzilla instance as:
  https://bugzilla.redhat.com/show_bug.cgi?id=667431

The handler part of the patch is based on a patch attached there by Federico Simoncelli; I don't know if he has signed a PSF contributor agreement, however the size of the patch is sufficiently small that I suspect it's not copyrightable, and I've somewhat rewritten it since; I wrote the unit test.

> It could be argued that this is incorrect, as it masks under-reads of
> stdin by the subprocess.  However I believe a sanely-written subprocess 
> ought to indicate success/failure back with its return code.

It seems quite orthogonal. The subprocess could have terminated successfully while not all of the stdin bytes were consumed; EPIPE informs you of the latter.

I agree they are orthogonal. The problem is that Popen.communicate doesn't support a way to ignore the exception and keep reading from stdout and sterr until the process dies.
When the child closes the stdin its (error/debug/info) messages on stout/sterr will be lost.
I am not just concerned about checking both the return-code and this exception, I am also worried about losing important information.

You are right, this is suboptimal. It would also be a behaviour change from currently, but it seems beneficial.

I'd argue that this is not a feature request but a bug.

I did some testing of this issue and the problem is that EPIPE is only generated sometimes depending on the time the process takes to finish, the size of the data sent, the underlying mechanism used (select vs poll) and the whether anything happens between the starting of the process and the communicate() call.

Here are some results (on my PC, I think some of these will vary on others):
With poll:
 [sys.executable, 'c', 'pass']- no error
 ['dd', 'option=bad'] - varies between EPIPE and no error
 ['dd', 'option=bad'], sleep(1) - EPIPE

With select:
 [sys.executable, 'c', 'pass']- EPIPE
 ['dd', 'option=bad'] - EPIPE
 ['dd', 'option=bad'], sleep(1) - EPIPE

Only stdin (neither select or poll):
 [sys.executable, 'c', 'pass']- no error (error in 2.7)
 ['dd', 'option=bad'] - no error (error in 2.7)
 ['dd', 'option=bad'], sleep(1) - EPIPE

(all of my tests appear to fail on Windows, they also generate EINVAL sometimes)

I think it's best to remove all this inconsistency and fix it so that EPIPE is never generated and then backport it to 2.7, 3.1, 3.2.

Attached is a patch which fixes it for poll, select, windows and adds two tests.

Marked #6457 as a duplicate. See #6457 for more discussion.

> I think it's best to remove all this inconsistency and fix it so that
> EPIPE is never generated and then backport it to 2.7, 3.1, 3.2.
> 
> Attached is a patch which fixes it for poll, select, windows and adds
> two tests.

The patch looks good to me.

New changeset c10d55c51d81 by Ross Lagerwall in branch '2.7':
Issue #10963: Ensure that subprocess.communicate() never raises EPIPE.
http://hg.python.org/cpython/rev/c10d55c51d81

New changeset 158495d49f58 by Ross Lagerwall in branch '3.1':
Issue #10963: Ensure that subprocess.communicate() never raises EPIPE.
http://hg.python.org/cpython/rev/158495d49f58

Committed, thanks.

self.stdout.close() also can fail with EPIPE or EINVAL if the stream is buffered (text streams are buffered). I.e. communicate() in text mode can loss the data. See also issue21619.

FTR Issue 26372 has been opened about Serhiy’s bug with stdin.close() raising EPIPE.