classification
Title: Subprocess searches special directories before PATH on Windows
Type: behavior Stage: needs patch
Components: Documentation, Library (Lib), Windows Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: berdario, docs@python, ezio.melotti
Priority: normal Keywords:

Created on 2013-05-26 19:15 by berdario, last changed 2017-02-04 20:53 by BreamoreBoy.

Messages (3)
msg190106 - (view) Author: (berdario) Date: 2013-05-26 19:15
To reproduce:
I installed 2 versions of python: python2.7 and python3.3, both in C:\
I then used the distribute-setup and get-pip script on both interpreters
http://python-distribute.org/distribute_setup.py
https://raw.github.com/pypa/pip/master/contrib/get-pip.py

I copied C:\Python33\Scripts\pip.exe to C:\Python33\pip.exe

And I set up the PATH system variable, to have C:\Python27\Scripts;C:\Python33\; at the beginning

When invoking python, it'll pick Python3.3, as expected, and then:

> python
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:57:17) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from shutil import which
>>> import os
>>> os.environ['PATH']
'C:\\Python27\\Scripts;C:\\Python33\\;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files\\Microsoft Windows Performance Toolkit\\;C:\\Program Files (x86)\\Bazaar;C:\\Program Files\\Mercurial\\;C:\\MinGW\\bin; C:\\Program Files (x86)\\Git\\cmd;C:\\Users\\Dario\\Documents\\WindowsPowerShell\\Modules\\Pscx\\Apps;C:\\Users\\Dario\\Applications\\bin;C:\\Users\\Dario\\Applications\\emacs\\bin'
>>> which('pip')
'c:\\python27\\scripts\\pip.exe'
>>> from subprocess import call
>>> call('pip --version'.split())
Cannot open C:\Python33\pip-script.py
2
>>>
msg190137 - (view) Author: (berdario) Date: 2013-05-27 09:56
I found out what's the problem, from the CreateProcess docs:

http://msdn.microsoft.com/en-us/library/ms682425.aspx

"""
If the file name does not contain a directory path, the system searches for the executable file in the following sequence:
The directory from which the application loaded.
The current directory for the parent process.
[...]
"""

This means that subprocess.Popen (and related functions) will always pick an executable in C:\PythonXX (if launched from the interpreter) or from C:\PythonXX\Scripts (if launched from a script/executable installed there) before looking into the PATH.

If we want to have the same behavior on *nix platforms and windows, I think that the only way this can be fixed, is by filtering the "executable" and "args" arguments through shutil.which (or a similar approach)

Otherwise, the difference in the executable path resolution should be documented.

(Either way, in my application, I'll have to reimplement shutil.which to be able to work on older python versions )
msg224347 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-30 21:41
Who is best placed to look at this, there's nobody active on the experts list for subprocess?
History
Date User Action Args
2017-02-04 20:53:01BreamoreBoysetnosy: - BreamoreBoy
2017-02-04 07:20:15martin.pantersettitle: Subprocess picks the wrong executable on Windows -> Subprocess searches special directories before PATH on Windows
2014-08-04 23:30:23ezio.melottisetassignee: docs@python

nosy: + docs@python
components: + Documentation
stage: needs patch
2014-07-30 21:41:43BreamoreBoysetversions: + Python 3.4, Python 3.5, - Python 3.3
nosy: + BreamoreBoy

messages: + msg224347

components: + Library (Lib), - Interpreter Core
2013-05-27 09:56:05berdariosetmessages: + msg190137
2013-05-26 19:18:07ezio.melottisetnosy: + ezio.melotti

versions: - Python 3.2
2013-05-26 19:15:42berdariocreate