classification
Title: Explain why it is recommended to pass args as a string rather than as a sequence If shell is True
Type: enhancement Stage:
Components: Documentation Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, eryksun, iMath, r.david.murray
Priority: normal Keywords:

Created on 2017-04-16 03:10 by iMath, last changed 2017-04-16 16:51 by r.david.murray.

Messages (5)
msg291733 - (view) Author: Philip Lee (iMath) Date: 2017-04-16 03:10
The doc here 
https://docs.python.org/3/library/subprocess.html#subprocess.Popen
says :
"If shell is True, it is recommended to pass args as a string rather than as a sequence."
but without explain why ? Please add the explanation !
while in
https://docs.python.org/3/library/subprocess.html#frequently-used-arguments
says:
"args is required for all calls and should be a string, or a sequence of program arguments. Providing a sequence of arguments is generally preferred, as it allows the module to take care of any required escaping and quoting of arguments (e.g. to permit spaces in file names). If passing a single string, either shell must be True (see below) or else the string must simply name the program to be executed without specifying any arguments."

In the case of shell =True ,  I found providing a sequence of arguments  rather than  a string argument can take the advantage of auto escaping and quoting of arguments (e.g. to permit spaces in file names) , so what is the advantage of pass args as a string rather than as a sequence as says in the doc when shell is True?
msg291734 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-04-16 04:19
Because passing a sequence to shell=True won't work on unix.  It only works more-or-less by accident on windows, even though the current docs kind of encourage it.  Yes, I think it would be good if these sentences were clarified.

See also issue 7839.
msg291735 - (view) Author: Eryk Sun (eryksun) * Date: 2017-04-16 05:14
In Unix, passing an args list with shell=True makes the first element the -c command. The remaining elements are arguments for the shell itself, which makes them $N variables. For example:

    >>> subprocess.call(['echo $0, $1', 'spam', 'eggs'], shell=True)
    spam, eggs

In Windows, subprocess.list2cmdline assumes VC++ rules for creating a command line from a list. cmd's parsing rules are very different -- e.g. '^' is the escape character, and double quotes also escape special characters, except not '%'. (Shockingly there is no way to escape '%' on the cmd.exe command line -- only in batch files by doubling it. If you think '^' escapes it, you're wrong. cmd looks for an environment variable containing the '^' character. By luck it's usually not found.) cmd's parsing is convoluted, to say the least. Now combine that with having to pass command lines for external programs in way that they survive cmd's parsing without also breaking how the external program parses its command line. This problem is intractable. Just use a string. For anything complicated you may have to experiment a few times to figure out some permutation that works. The best option is to not use the shell at all.
msg291755 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-04-16 16:49
Note the subtlety here on unix:

rdmurray@pydev:~/python/p36[3.6]>cat temp.sh
#!/bin/bash
echo $0
echo $1

>>> subprocess.call(['./temp.sh', 'spam', 'eggs'], shell=True)
./temp.sh


0
>>> subprocess.call(['./temp.sh $0 $1', 'spam', 'eggs'], shell=True)
./temp.sh
spam
eggs
0
msg291756 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-04-16 16:51
Woops, cut and paste error, there should have been an "echo $2" line in that script as well.
History
Date User Action Args
2017-04-16 16:51:02r.david.murraysetmessages: + msg291756
2017-04-16 16:49:50r.david.murraysetmessages: + msg291755
2017-04-16 05:14:03eryksunsetnosy: + eryksun
messages: + msg291735
2017-04-16 04:19:45r.david.murraysetnosy: + r.david.murray
messages: + msg291734
2017-04-16 03:10:04iMathcreate