classification
Title: distutils.spawn: find_executable() Fails To Find Many Executables on Windows
Type: Stage: resolved
Components: Distutils Versions: Python 3.8
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: dstufft, eric.araujo, eryksun, steve.dower, tbpassin, vstinner
Priority: normal Keywords:

Created on 2020-01-08 15:53 by tbpassin, last changed 2021-02-03 18:07 by steve.dower. This issue is now closed.

Messages (8)
msg359602 - (view) Author: Thomas Passin (tbpassin) Date: 2020-01-08 15:53
On Windows, find_executable() in distutils.spawn may fail to find executables that it ought to.  This is because the PATH environmental variable no longer includes %ProgramFiles% and %ProgramFiles(x86)%.  At least, that is the case on my brand new Windows 10 Computer running Windows 10 Pro.  In the past, I'm fairly sure these directories were always included on the PATH.

Some programs add their install directory to the Windows PATH, but many don't.  For example, on my new computer, Pandoc added itself to the PATH but EditPlus and Notepad++ did not.  So

    >>> find_executable('pandoc')
    'C:\\Program Files\\Pandoc\\pandoc.exe'
but
    >>> find_executable('editplus')   # no result
    >>> find_executable('notepad++')  # no result

I suggest that in Windows, find_executable() should check for and add the %ProgramFiles% and %ProgramFiles(x86)% directories to the system PATH before executing its search.
msg359603 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-01-08 15:55
find_executable() rely on the PATH environment variable:

def find_executable(executable, path=None):
    """Tries to find 'executable' in the directories listed in 'path'.

    A string listing directories separated by 'os.pathsep'; defaults to
    os.environ['PATH'].  Returns the complete filename or None if not found.
    """

You may add manually %ProgramFiles% and %ProgramFiles(x86)% to your PATH if they aren't already there.
msg359605 - (view) Author: Thomas Passin (tbpassin) Date: 2020-01-08 16:27
Well, yes, I realize that I could add the directories.  But random users of find_executible may not realize that.

Actually, it's probably a bit more complex than just adding the top-level directories %ProgramFiles% and %ProgramFiles(x86)%, because  find_executable probably doesn't do a recursive search.  In that the best tactic would be to look in those top-level directories for a directory with the same name as the executable.  If one is found, then look in it for the actual file.

For example, on my computer, inkscape.exe is at %ProgramFiles%\Inkscape
inkscape.exe, and that pattern is very common.

This tactic would find many of the otherwise unfound executables, and still execute quickly.
msg359614 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2020-01-08 17:25
> In that the best tactic would be to look in those top-level 
> directories for a directory with the same name as the executable.

It may be a common pattern, but such a guess is not reliable. The needed executable may be in an unrelated directory at an arbitrary depth in the tree. 

There are reliable ways to find an executable in Windows other than searching PATH -- assuming the application wants to be found in a reliable way. For example, we can use winreg to search the subkey names of "[HKCU|HKLM]\Software\Microsoft\Windows\CurrentVersion\App Paths" for the given executable name (e.g. "notepad++.exe"). The default value of each subkey is the fully-qualified path of the executable.
msg359617 - (view) Author: Thomas Passin (tbpassin) Date: 2020-01-08 17:50
That's so, and with a bit more complexity might be worth doing.  At least, it would be good to have *some* means to find more the the executable files one would expect to find.
msg359644 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2020-01-09 01:45
The problem is that distutils.spawn.find_executable is an internal API, never meant for general usage.

Only minimal change happens to distutils these days, partly because the codebase is full of dark corners and weird interactions (so a fix can easily break someone else’s setup.py script), partly because few core developers have the expertise and willingess to write patches or review PRs.

So the question is: is the problem you reported a problem that happens when people write setup.py scripts, or when using the function for something else?  If it’s the latter, I’m afraid the safe course of action would be to change nothing, and recommend people use truly general-use functions like shutil.which.
msg359647 - (view) Author: Thomas Passin (tbpassin) Date: 2020-01-09 02:34
I came across it while working on some code running in the Leo editor.  I had no idea it was not really public; I just assumed that someone knew something I didn't.  Then I discovered that it couldn't find some files I  thought it clearly should, and wrote the issue, thinking it might help someone else who didn't know the limitations.

But from what you say, I see it will be better to use another way, and to leave this bit alone.  Thank you.
msg386260 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-02-03 18:07
Distutils is now deprecated (see PEP 632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.

If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools
History
Date User Action Args
2021-02-03 18:07:54steve.dowersetstatus: open -> closed

nosy: + steve.dower
messages: + msg386260

resolution: out of date
stage: resolved
2020-01-09 02:34:08tbpassinsetmessages: + msg359647
2020-01-09 01:45:04eric.araujosetmessages: + msg359644
2020-01-08 17:50:54tbpassinsetmessages: + msg359617
2020-01-08 17:25:22eryksunsetnosy: + eryksun
messages: + msg359614
2020-01-08 16:27:46tbpassinsetmessages: + msg359605
2020-01-08 15:55:48vstinnersettitle: find_executable() Fails To Find Many Executables on Windows -> distutils.spawn: find_executable() Fails To Find Many Executables on Windows
2020-01-08 15:55:40vstinnersetnosy: + vstinner
messages: + msg359603
2020-01-08 15:53:12tbpassincreate