Author vstinner
Recipients gregory.p.smith, haizaar, martin.panter, miss-islington, twouters, vstinner, xtreak
Date 2019-09-11.10:55:18
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1568199318.57.0.189219775598.issue37424@roundup.psfhosted.org>
In-reply-to
Content
On Windows, the following pattern _can_ hang:

proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
try:
  return proc.communicate(timeout=1.0)
except TimeoutExpired:
  proc.kill()
  return proc.communicate() # <== HERE

Even if the first child process is killed, communicate() waits until the pipe is closed. If the child process spawned a 3rd process before being killed, the second .communicate() calls hangs until the 3rd process exit or close its stdout.

I'm not sure if subprocess.run() should do anything about this case, but it was at least for your information.

I'm fighting against this issue in bpo-37531.

IMHO it's an issue of the Windows implementation of Popen.communicate(): it's implemented with a blocking call to stdout.read() run in a thread. The thread cannot be interrupted in any way and will until complete once stdout is closed.

Again, if the direct child process spawned other processes, stdout is only closed in the main parent process once all child processes exit or at least closed their stdout.

Maybe another issue should be opened to avoid blocking with the following pattern on Windows:

proc.kill()
proc.communicate()
History
Date User Action Args
2019-09-11 10:55:18vstinnersetrecipients: + vstinner, twouters, gregory.p.smith, martin.panter, miss-islington, xtreak, haizaar
2019-09-11 10:55:18vstinnersetmessageid: <1568199318.57.0.189219775598.issue37424@roundup.psfhosted.org>
2019-09-11 10:55:18vstinnerlinkissue37424 messages
2019-09-11 10:55:18vstinnercreate