classification
Title: subprocess.run() timeout not working with grandchildren and stdout=PIPE
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder: subprocess.run with stderr connected to a pipe won't timeout when killing a never-ending shell commanad
View: 30154
Assigned To: Nosy List: Martin Ritter, martin.panter, vstinner
Priority: normal Keywords:

Created on 2017-11-03 15:01 by Martin Ritter, last changed 2018-07-22 08:38 by martin.panter. This issue is now closed.

Files
File name Uploaded Description Edit
test_killsub.py Martin Ritter, 2017-11-03 15:01 Minimal example to reproduce timeout problem
Messages (3)
msg305487 - (view) Author: Martin Ritter (Martin Ritter) Date: 2017-11-03 15:01
Hi,

I tried to use 

    subprocess.run(..., stdout=subprocess.PIPE, timeout=N)

to run some test scripts with a given timeout. 

This works as expected with simple scripts. However if the script itself creates other children which write to stdout then `subprocess.run()` seems to wait for all of the children to finish. I've attached a minimal example.

I looked into subprocess.py and `subprocess.run()` calls `process.communicate()` again without timeout when handling the TimeoutExpired exception which then in turn waits for the pipes to be closed by all children.

If communicate() would check if the process is still alive while waiting for output and close the pipes once the process has finished the timeout feature should work as expected and descendants would get a SIGPIPE when writing to stdout/stderr.
msg305529 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2017-11-04 00:44
This proposal sounds like a race condition. Closing the output pipe as a child exits means you risk missing recent output. On the other hand, if you don’t care about the output any more, close the pipe first and then wait for the child.

Related discussions:

Issue 30154: Similar problem, but the grandchild does not write any output
Issue 26534: kill_group=True option for timeout
Issue 31447: Windows-specific (?)
msg322137 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2018-07-22 08:38
Closing in faviour of Issue 30154, which suggests documentation or adjusting the timeout implementation. There is also Issue 26534 proposing a new “kill_group” option when using the timeout feature.
History
Date User Action Args
2018-07-22 08:38:36martin.pantersetstatus: open -> closed
superseder: subprocess.run with stderr connected to a pipe won't timeout when killing a never-ending shell commanad
messages: + msg322137

resolution: duplicate
stage: resolved
2017-11-06 16:52:32vstinnersetnosy: + vstinner
2017-11-04 00:44:19martin.pantersetnosy: + martin.panter
messages: + msg305529
2017-11-03 15:01:32Martin Rittercreate