classification
Title: subprocess doesn't pass arguments correctly on Linux when shell=True
Type: behavior Stage:
Components: Library (Lib) Versions: Python 2.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: benhoyt, berwyn, davidfraser (3)
Priority: Keywords patch

Created on 2009-08-12 08:33 by davidfraser, last changed 2009-11-16 08:44 by davidfraser.

Files
File name Uploaded Description Edit Remove
subprocess-shell-args-2.6.patch davidfraser, 2009-08-14 08:21 Patch against Python 2.6.2 without unneccessary args
Messages (5)
msg91492 - (view) Author: David Fraser (davidfraser) Date: 2009-08-12 08:33
(from
http://stackoverflow.com/questions/1253122/why-does-subprocess-popen-with-shelltrue-work-differently-on-linux-vs-windows/1254322)

When using subprocess.Popen(args, shell=True) to run "gcc --version"
(just as an example), on Windows we get this:

>>> from subprocess import Popen
>>> Popen(['gcc', '--version'], shell=True)
gcc (GCC) 3.4.5 (mingw-vista special r3) ...

So it's nicely printing out the version as I expect. But on Linux we get
this:

>>> from subprocess import Popen
>>> Popen(['gcc', '--version'], shell=True)
gcc: no input files

Because gcc hasn't received the --version option.

The docs don't specify exactly what should happen to the args under
Windows, but it does say, on Unix, "If args is a sequence, the first
item specifies the command string, and any additional items will be
treated as additional shell arguments." IMHO the Windows way is better,
because it allows you to treat Popen(arglist) calls the same as
Popen(arglist, shell=True) ones.

The strange implementation is actually the UNIX one, which does the
following (where each space separates a different argument):

/bin/sh -c gcc --version

It looks like the correct implementation (at least on Linux) would be:

/bin/sh -c "gcc --version" gcc --version

Since this would set the command string from the quoted parameters, and
pass the other parameters successfully.

From the sh man page section for -c:

    Read commands from the command_string operand instead of from the
standard input. Special parameter 0 will be set from the command_name
operand and the positional parameters ($1, $2, etc.) set from the
remaining argument operands.

This patch seems to fairly simply do the trick, as well as testing it.
msg91510 - (view) Author: Ben Hoyt (benhoyt) Date: 2009-08-12 23:18
Oops, didn't intend to change the type, changing back.
msg91511 - (view) Author: Berwyn (berwyn) Date: 2009-08-13 01:17
I agree with the patch, and it works fine:
    /bin/sh -c "gcc --version" gcc --version

But I begin to wonder whether bash has a bug since bash only seems to
observe the string, not the parameters after it.  For example:
    $ bash -c "./printargs.py abc" def ghi jkl
produces:
    ['./printargs.py', 'abc']

where printargs.py is:
    #!/usr/bin/python
    import sys
    print sys.argv

Just something to be aware of.
msg91542 - (view) Author: David Fraser (davidfraser) Date: 2009-08-14 08:21
Ah, upon closer inspection - the special parameters $0 $1 $2 etc
mentioned in the sh docs refer to parameters within the command string,
so that:
   sh -c 'echo $2 $0 $1' run for "the people"
produces:
   the people run for

So the correct patch would be to leave out the extra parameters and just
have the quoted string...

The question then becomes, is simple " ".join(args) sufficient or should
there be some quoting of them... attaching a patch in the meantime
although that needs to be resolved.
msg95329 - (view) Author: David Fraser (davidfraser) Date: 2009-11-16 08:44
This is closely related to http://bugs.python.org/issue2304
History
Date User Action Args
2009-11-16 08:44:43davidfrasersetmessages: + msg95329
2009-08-14 08:21:31davidfrasersetfiles: - current-3.patch
2009-08-14 08:21:21davidfrasersetfiles: - current-2.6.patch
2009-08-14 08:21:02davidfrasersetfiles: + subprocess-shell-args-2.6.patch

messages: + msg91542
2009-08-13 01:17:07berwynsetnosy: + berwyn
messages: + msg91511
2009-08-12 23:18:02benhoytsettype: feature request -> behavior
messages: + msg91510
2009-08-12 23:16:38benhoytsetnosy: + benhoyt
type: behavior -> feature request
2009-08-12 08:34:45davidfrasersetfiles: + current-3.patch
2009-08-12 08:33:56davidfrasercreate