classification
Title: Extend subprocess.kill to be able to kill process groups
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: subprocess.run timeout does not function if shell=True and capture_output=True
View: 37424
Assigned To: Nosy List: Ernst.Sjöstrand, amaury.forgeotdarc, brian.curtin, erickt, eryksun, fangyizhou, giampaolo.rodola, gregory.p.smith, guettli, haizaar, neologix, vstinner, xuanji
Priority: normal Keywords: patch

Created on 2009-01-31 02:50 by erickt, last changed 2019-07-01 00:06 by gregory.p.smith. This issue is now closed.

Files
File name Uploaded Description Edit
killpg.patch erickt, 2009-01-31 02:50 review
Messages (8)
msg80850 - (view) Author: Erick Tryzelaar (erickt) Date: 2009-01-31 02:50
It would be really handy to add a way to portably kill process groups of 
a process spawned with subprocess.Popen. My project starts a process, 
which then starts other long running processes. Because of this process 
tree, there's no way for me to portably kill all of those processes. 
Would it be possible to extend subprocess with these features? I did 
find someone who's already implemented this [1], and it's working well 
in my project. There was some discussion about adding this, but it seems 
no one actually committed a patch to implement it.

Anyway, I've attached a patch against python3.0's svn branch that 
implements this. I don't have access to windows, so I'm not able to make 
sure that the TerminateJobObject is correct, and works.

[1]: http://benjamin.smedbergs.us/blog/2006-12-11/killableprocesspy/
[2]: http://mail.python.org/pipermail/python-list/2008-April/487588.html
msg81076 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-02-03 18:07
I suspect that on Windows, TerminateJobObject won't work with a handle
returned by CreateProcess.
A solution could look like 
    http://www.mobzystems.com/code/killprocesstree.aspx
msg98087 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-01-21 00:29
> I suspect that on Windows, TerminateJobObject won't work 
> with a handle returned by CreateProcess.

TerminateJobObject works with CreateJobObject and AssignProcessToJobObject. The following code (from your patch) should call AssignProcessToJobObject (and CreateJobObject if there is no job yet):

+                    if not mswindows:
+                        os.setpgid(0, 0)
msg107615 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2010-06-12 00:22
erickt: can you fix your patch?
msg108062 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2010-06-17 21:10
A strong +1 on adding this feature.

I was wondering... what if process A runs a subprocess B which runs a subprocess C. Is C still considered a children of A and gets killed as well?

Does os.setpgid() takes care of such a thing? What about Windows instead?
msg134500 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-04-26 21:09
Note that the setpgid creation part is now somewhat redundant with Popen's start_new_session flag (which calls setsid). Also, this should probably be an option, since with that patch every subprocess is in its own process group.

> I was wondering... what if process A runs a subprocess B which runs a
> subprocess C. Is C still considered a children of A and gets killed as
> well?

No.
When setpgid/setsid is called, a new group is created, so process C will be not be part of the same group as B.
msg292233 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2017-04-24 18:30
Using a Windows job object should be paired with the creation flag CREATE_SUSPENDED. Callers may also need CREATE_BREAKAWAY_FROM_JOB, but setting that creation flag shouldn't be integrated into Popen.

The child has to be created suspended to ensure it doesn't spawn another process and exit before it's added to the job. Once it's in the job, call ResumeThread to start it. 

On Windows Vista and 7, the child may need to break away from Python's current job, if allowed. These older versions of Windows don't implement nested jobs, so adding the child to a job will fail if it's already in one. The job used by py.exe for python.exe isn't a problem in this case since it's configured for child processes to automatically break away, but python.exe may have been started directly and added to a job that's not configured for silent breakaway.
msg346945 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2019-07-01 00:06
this is part of a chain of other related issues.  the interesting thing in the patch in here is the potential feature to have send_signal/terminate/kill be able to optionally kill a process group.

the logic as is in the patch in this issue is dangerous as often the calling process _is_ the pgrp leader and would be killed.  we _never_ want the subprocess signaling methods to kill the current process as that would be surprising to users.
History
Date User Action Args
2019-07-01 00:06:10gregory.p.smithsetstatus: open -> closed

nosy: + gregory.p.smith
messages: + msg346945

stage: patch review -> resolved
2019-07-01 00:04:26gregory.p.smithsetsuperseder: subprocess.run timeout does not function if shell=True and capture_output=True
resolution: duplicate
2019-06-30 22:55:07gregory.p.smithsetpull_requests: - pull_request7893
2019-06-27 13:20:42haizaarsetnosy: + haizaar
2018-07-20 20:26:43ammar2setpull_requests: + pull_request7893
2018-02-16 13:03:38fangyizhousetnosy: + fangyizhou
2017-04-24 18:30:07eryksunsetnosy: + eryksun

messages: + msg292233
versions: + Python 3.7, - Python 3.3
2017-04-24 11:47:07martin.panterlinkissue26534 dependencies
2011-04-26 21:09:52neologixsetnosy: + neologix
messages: + msg134500
2011-04-26 15:01:49xuanjisetnosy: + xuanji
2011-04-18 13:06:25giampaolo.rodolasetversions: + Python 3.3, - Python 3.0
2011-04-18 13:01:55Ernst.Sjöstrandsetnosy: + Ernst.Sjöstrand
2010-06-17 21:10:58giampaolo.rodolasetmessages: + msg108062
2010-06-12 00:22:40vstinnersetmessages: + msg107615
2010-06-09 17:45:29giampaolo.rodolasetnosy: guettli, amaury.forgeotdarc, vstinner, giampaolo.rodola, erickt, brian.curtin
2010-01-21 00:29:30vstinnersetnosy: + vstinner
messages: + msg98087
2010-01-20 11:49:07guettlisetnosy: + guettli
2010-01-13 02:03:33brian.curtinsetpriority: normal
stage: patch review
2009-12-31 01:25:04brian.curtinsetnosy: + brian.curtin
2009-02-03 18:07:02amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg81076
2009-01-31 19:54:15giampaolo.rodolasetnosy: + giampaolo.rodola
2009-01-31 02:50:59ericktcreate