classification
Title: Subprocess does not find executable on Windows if it is PATH with quotes
Type: behavior Stage: resolved
Components: Library (Lib), Windows Versions: Python 3.7, Python 3.6, Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: tim.golden Nosy List: BreamoreBoy, Eli_B, eryksun, pekka.klarck, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2013-01-24 13:09 by pekka.klarck, last changed 2020-03-18 11:19 by eryksun. This issue is now closed.

Messages (4)
msg180520 - (view) Author: Pekka Klärck (pekka.klarck) Date: 2013-01-24 13:09
If you add a directory into PATH on Windows so that the directory is in quotes, subprocess does not find executables in it. They are found by the operating system, though, at least when run on the command prompt.

To reproduce:

C:\>python --version
Python 2.7.3
C:\>type test\script.bat
@echo off
echo hello
C:\>set ORIG=%PATH%
C:\>set PATH=%ORIG%;test
C:\>script.bat
hello
C:\>python -c "from subprocess import call; call('script.bat')"
hello
C:\>set PATH=%ORIG%;"test"
C:\>script.bat
hello
C:\>python -c "from subprocess import call; call('script.bat')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\subprocess.py", line 493, in call
    return Popen(*popenargs, **kwargs).wait()
  File "C:\Python27\lib\subprocess.py", line 679, in __init__
    errread, errwrite)
  File "C:\Python27\lib\subprocess.py", line 896, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified


I don't think you ever need those quotes, even if the directory would have spaces, but it is very confusing that the behavior is different on the command prompt and with subprocess. Additionally, Linux does not suffer from this problem:

$ python --version
Python 2.7.3
$ cat test/script.sh 
#!/bin/sh
echo "hello"
$ PATH=$PATH:test script.sh
hello
$ PATH=$PATH:test python -c "from subprocess import call; call('script.sh')"
hello
$ PATH=$PATH:"test" script.sh
hello
$ PATH=$PATH:"test" python -c "from subprocess import call; call('script.sh')"
hello
msg222891 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-12 22:29
@Pekka sorry about the delay in getting back to you.

I can confirm that this is still in issue in 3.4.1 and 3.5.0a0.
msg235393 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2015-02-04 18:06
Under the covers, subprocess is calling CreateProcess so there's really not very much we can do here, short of writing our own PATH-handling.

As a matter of fact, passing shell=True will produce the desired effect. Since the only thing this does is to run the process under cmd.exe I assume that cmd.exe itself adds some kind of PATH handling of its own.

I recommend closing as "won't fix".
msg364512 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2020-03-18 11:19
Why are new versions getting added to an issue that was closed 5 years ago? 

That said, I'd like to clarify that this was not and is not a bug. It happens that the CMD shell strips quotes out, but that doesn't make it valid. PATH in Windows is delimited by semicolons, not spaces, so paths with spaces should never be quoted. In particular, WINAPI SearchPathW, which CreateProcessW calls, leaves quotes in the directory name when testing whether a file is in the directory.

Also, the conclusion that Linux doesn't suffer from this problem is incorrect. The shell command that was used actually strips the quotes out as part of command-line pre-processing, i.e. `PATH=$PATH:"test"` results in the same PATH value as `PATH=$PATH:test`. But it will fail to find "script.sh" if you escape the quotes to make the shell add them literally to PATH, e.g. `PATH=$PATH:\"test\"`.
History
Date User Action Args
2020-03-18 11:19:02eryksunsetresolution: wont fix -> not a bug

messages: + msg364512
nosy: + eryksun
2020-03-18 10:10:08Oskar Perssonsetversions: + Python 3.6, Python 3.7
2015-02-08 12:53:07tim.goldensetstatus: open -> closed
resolution: wont fix
stage: resolved
2015-02-04 18:06:13tim.goldensetnosy: + zach.ware, tim.golden, steve.dower
messages: + msg235393

assignee: tim.golden
components: + Windows
2015-02-04 17:50:24Eli_Bsetnosy: + Eli_B
2014-07-12 22:29:41BreamoreBoysetversions: + Python 2.7, Python 3.4, Python 3.5
nosy: + BreamoreBoy

messages: + msg222891

components: + Library (Lib)
type: behavior
2013-01-24 13:09:21pekka.klarckcreate