classification
Title: Use user shell in subprocess
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: subrocess.Popen needs /bin/sh but Android only has /system/bin/sh
View: 16255
Assigned To: Nosy List: Jan Studený, eryksun, r.david.murray
Priority: normal Keywords:

Created on 2015-08-23 21:47 by Jan Studený, last changed 2015-08-23 23:16 by eryksun. This issue is now closed.

Messages (4)
msg249023 - (view) Author: Jan Studený (Jan Studený) Date: 2015-08-23 21:47
According to POSIX specification the pathname of user shell is stored in SHELL (environmental variable, see http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08) so I think that is good idea to use that pathname instead of hardcoded one.
Change is only in one line of subprocess package to use SHELL environmental variable and use hardcoded shell pathname as fallback.

lines 1431-1433
```
            if shell:
                args = ["/bin/sh", "-c"] + args
                if executable:
```

to

```
            if shell:
                args = [os.environ.get("SHELL","/bin/sh"), "-c"] + args
                if executable:
```
msg249025 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-08-23 22:30
Thanks for the suggestion, but that would make programs using subprocess non-portable.  There is an open issue to use the *default* shell instead of hard coding it (because the sh-alike is not at /bin/sh on, eg, Android), but using the user shell would break lots of programs.  Your suggestion was brought up by a core dev (and rejected) in that issue (issue 16255)
msg249027 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2015-08-23 23:03
I expect Popen's shell=True option to use the same shell as os.system.  The POSIX spec for the [system function][1] requires running sh, as follows:

    execl(<shell path>, "sh", "-c", command, (char *)0);

glibc uses "/bin/sh" for the shell path. Modifying the SHELL environment variable doesn't affect the behavior of os.system. I would be surprised if Popen's shell=True had been designed like that. Thankfully it's too late to change this since it could break existing code. ;-)

The story is different on Windows. The CRT's system() prefers the shell that's set in the ComSpec environment variable. So on Windows Popen uses os.environ.get("COMSPEC", "cmd.exe").

[1]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html
msg249028 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2015-08-23 23:16
Here's the rationale given for ignoring SHELL in POSIX system():

    One reviewer suggested that an implementation of system() might
    want to use an environment variable such as SHELL to determine
    which command interpreter to use. The supposed implementation
    would use the default command interpreter if the one specified
    by the environment variable was not available. This would allow
    a user, when using an application that prompts for command lines
    to be processed using system(), to specify a different command
    interpreter. Such an implementation is discouraged. If the
    alternate command interpreter did not follow the command line
    syntax specified in the Shell and Utilities volume of
    POSIX.1-2008, then changing SHELL would render system()
    non-conforming. This would affect applications that expected the
    specified behavior from system(), and since the Shell and
    Utilities volume of POSIX.1-2008 does not mention that SHELL
    affects system(), the application would not know that it needed
    to unset SHELL.
History
Date User Action Args
2015-08-23 23:16:41eryksunsetmessages: + msg249028
2015-08-23 23:03:32eryksunsetnosy: + eryksun
messages: + msg249027
2015-08-23 22:30:27r.david.murraysetstatus: open -> closed

superseder: subrocess.Popen needs /bin/sh but Android only has /system/bin/sh

nosy: + r.david.murray
messages: + msg249025
resolution: duplicate
stage: resolved
2015-08-23 21:47:42Jan Studenýcreate