This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Popen(..., shell=True,...) should allow simple access to the command's PID
Type: enhancement Stage:
Components: Extension Modules Versions:
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: belopolsky, georg.brandl, wolfy
Priority: normal Keywords:

Created on 2009-01-06 05:06 by wolfy, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (3)
msg79233 - (view) Author: (wolfy) Date: 2009-01-06 05:40
When using proc = Popen() with the parameter "shell=True", the returned 
proc-object's Process ID (PID) obtained by reading "proc.pid" is the 
one of the shell that was created.

This gets very cumbersome, when the newly created process shall be 
signalled later on.
It is very rare that one needs to signal the shell instead of the newly 
started executable.

Moreover, sending a signal to the shell does (for some reason) not get 
propagated to the child (=the executable/command), so the shell is quit/
terminated/killed/whatever and the command's thread is kept alive.

Using 'shell=False' is not an option in case the originating process 
has more code to execute, e.g. in order to communicate with the newly 
started process via pipes (i.e. using proc.stdout.readline() or 
similar), because in that case Python uses os.execvp() for the command, 
which re-uses (and therefore blocks) the current thread of control in 
which the Popen was issued.

Starting another thread with fork() is also not an option, as the Popen 
will be executed in a different thread, which makes it difficult to 
pass the reference to the proc-object back into the original thread 
where it is needed for the inter-process-communication.

All examples I found on the net which use pipes to communicate between 
current and newly started command use of course 'shell=True', I did not 
however find any simple way to get the child's PID (aside from issuing 
a 'ps --ppid <PPID> -o pid=' and reading it's output, as I did, which 
is far from elegant).

So, there should be one of these alternative possibilities/behaviours 
in case of 'Popen(...,shell=True...)':

1) proc gets a new field or method child_pid/child_pid() which returns 
the command's PID. proc.pid behaves as before
2) Signalling the shell's process notifies all children with the same 
signal. proc.pid behaves as before 
3) proc.pid returns the command's PID (instead of the shell's)
4) There is some simple command to query all the childs PIDs of a 
process when passing in the PPID (parent PID). proc.pid behaves as 
before

...with 1) being the most up to 4) being the least desireable.

I hope I did not overlook any such possibility, but searching around on 
the net and in the tracker turned up nothing useful.

I really do think this should be more intuitive.
msg79235 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2009-01-06 08:28
This really belongs to python-list rather than the tracker.  It is not 
correct that with shell=False Popen reuses the thread of control of the 
calling process.  You seem to be confusing blocking and reusing the 
thread of control.  Popen always creates a new thread of control using a 
fork() call.  The  proc.stdout.readline() will block until the process 
produces a full line of output, which is what anyone would expect.

It is not clear what you are trying to achieve.  Here is an example 
session:

>>> from subprocess import *
>>> p = Popen(['sleep', '10000'])
>>> p.kill()
>>> p.wait()
-9

or using a different signal (SIGINT = 2):

>>> p = Popen(['sleep', '10000'])
>>> p.send_signal(2)
>>> p.wait()
-2

The wait function blocks until the process terminates and returns the 
status (negative of the signal number in case of exit on signal).  What 
else are you trying to achieve?

Your proposals don't make much sense:

1) "child_pid()"  - what should it return if the shell command spawns 
more than one process? For example if the command contains |'s or 
multiple &'s?

2) "Signalling the children of the shell" - read the manual page for 
your shell.  Foreground processes receive most of the signals that way.  
Background processes only get a SIGHUP.

3) "proc.pid returns the command's PID" - there is no such thing: a 
command can create any number of processes.

4) "simple command to query all the childs PIDs" - without 
reimplementing shell in python to parse the command string, it is 
impossible to tell which subprocesses are spawned by a given Popen call.  
A function that returns all children of the given process can be 
written, but will be expensive on most systems because it will require a  
search over the entire process table.

In short, this report is invalid.
msg79240 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-01-06 09:19
As Alexander says: a shell command can spawn any number of subprocesses,
or none at all (if e.g. only shell builtins are called).
History
Date User Action Args
2022-04-11 14:56:43adminsetgithub: 49105
2009-01-06 09:19:06georg.brandlsetstatus: open -> closed
resolution: wont fix
messages: + msg79240
nosy: + georg.brandl
2009-01-06 08:28:34belopolskysetnosy: + belopolsky
messages: + msg79235
components: + Extension Modules
2009-01-06 05:41:56wolfysettitle: Popen with Shell=True should return the command's PID, not the shell's -> Popen(..., shell=True,...) should allow simple access to the command's PID
2009-01-06 05:40:24wolfysettype: enhancement
messages: + msg79233
title: Popen with Shell=True should return the command -> Popen with Shell=True should return the command's PID, not the shell's
2009-01-06 05:06:10wolfycreate