Issue40467
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2020-05-01 13:34 by anishathalye, last changed 2022-04-11 14:59 by admin.
Messages (3) | |||
---|---|---|---|
msg367844 - (view) | Author: Anish Athalye (anishathalye) | Date: 2020-05-01 13:34 | |
On Windows, using subprocess.call() and specifying both shell=True and the executable='...' keyword arguments produces an undesirable result when the specified shell is a POSIX-like shell rather than the standard cmd.exe. I think the documentation is unclear on the semantics of Popen() when both shell=True and executable= are specified on Windows. It does say the following about POSIX systems: > If shell=True, on POSIX the executable argument specifies a replacement shell > for the default /bin/sh. But the documentation doesn't say anything about Windows in this scenario, so I'm not sure if this is a bug, or if it's undefined behavior. Concretely, here's an example program that fails due to this: import subprocess bash = 'C:\\Program Files\\Git\\usr\\bin\\bash.exe' subprocess.call('f() { echo test; }; f', shell=True, executable=bash) It prints out this mysterious-looking error: /c: /c: Is a directory Tracing this into subprocess.py, it looks like it's because the executable bash (as specified) is being called with the argv that's approximately ['cmd.exe', '/c', 'f() { echo test; }; f'] (and the program being launched is indeed bash). Bash doesn't expect a '/c' argument, it wants a '-c' there. The problematic code in subprocess.py is here: https://github.com/python/cpython/blob/1def7754b7a41fe57efafaf5eff24cfa15353444/Lib/subprocess.py#L1407 If the '/c' is replaced with a '-c', the example program above works (bash doesn't seem to care that it's called with an argv[0] that doesn't make sense, though presumably that should be fixed too). I'm not sure how this could be fixed. It's unclear when '/c' should be used, as opposed to '-c'. Doing it based on the contents of the executable= argument or the SHELL environment variable or COMSPEC might be fragile? I couldn't find much about this online, but I did find one project (in Ruby) that seems to have run into a similar problem. See https://github.com/kimmobrunfeldt/chokidar-cli/issues/15 and https://github.com/kimmobrunfeldt/chokidar-cli/pull/16. At the very least, even if this isn't fixed / can't be fixed, it might be nice if it's possible to give some sort of useful warning/error when this happens -- perhaps say that specifying both shell=True and executable="..." isn't supported on Windows? I ran into this issue while while debugging an issue in a project of mine. In case the additional context is useful, here is the discussion: https://github.com/anishathalye/dotbot/issues/219 |
|||
msg407644 - (view) | Author: Joe Cool (snoopyjc) | Date: 2021-12-04 06:55 | |
Proposed solution: if comspec.endswith('sh.exe') or comspec.endswith('sh'): # issue 40467 args = '{} -c "{}"'.format (comspec, args) # issue 40467 else: # issue 40467 args = '{} /c "{}"'.format (comspec, args) |
|||
msg407647 - (view) | Author: Eryk Sun (eryksun) * | Date: 2021-12-04 09:08 | |
> it might be nice if it's possible to give some sort of useful > warning/error when this happens -- perhaps say that specifying > both shell=True and executable="..." isn't supported on Windows? The `shell` parameter is documented as follows for Windows: On Windows with shell=True, the COMSPEC environment variable specifies the default shell. The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable. It wouldn't hurt to clarify that the COMSPEC shell has to support `/c`. This is required by CreateProcessW(), which uses COMSPEC to run BAT and CMD files. The discussion about using `executable` with `shell` could be extended for Windows. But this would also require new behavior. For example: Original: if shell: startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW startupinfo.wShowWindow = _winapi.SW_HIDE comspec = os.environ.get("COMSPEC", "cmd.exe") args = '{} /c "{}"'.format (comspec, args) Proposed: if shell: startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW startupinfo.wShowWindow = _winapi.SW_HIDE if executable is not None: cmd = executable else: cmd = os.path.normpath(os.environ.get("COMSPEC", "cmd.exe")) if "\\" in cmd: executable = cmd args = '"{}" /c "{}"'.format(cmd, args) > if comspec.endswith('sh.exe') or comspec.endswith('sh'): > args = '{} -c "{}"'.format (comspec, args) sh is not a valid COMSPEC shell. To use sh automatically, subprocess would have to support and prefer the SHELL [1] environment variable in Windows -- and in POSIX for that matter. --- [1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_03 |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:30 | admin | set | github: 84647 |
2021-12-04 09:14:59 | eryksun | set | versions: + Python 3.10, Python 3.11, - Python 3.7 |
2021-12-04 09:08:33 | eryksun | set | nosy:
+ eryksun messages: + msg407647 |
2021-12-04 06:55:34 | snoopyjc | set | nosy:
+ snoopyjc messages: + msg407644 |
2020-05-01 13:34:17 | anishathalye | create |