This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Inconsistent environment in Windows using "Open With"
Type: behavior Stage: resolved
Components: Installation, Windows Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: Nosy List: busfault, eryksun, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2016-04-26 16:01 by busfault, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (6)
msg264315 - (view) Author: Tom Middleton (busfault) Date: 2016-04-26 16:01
I have found that the execution of python scripts is inconsistent from the following methods:
From Explorer:

1) Right-Click->Open with->python.exe
2) Right-Click->Open (assuming python.exe being the "default" application)
3) Right-Click->Open with->"Choose default program..."->python.exe
4) Double-Click the script
from the command prompt:
4) python <scriptname>
5) <scriptname>

Of those listed, #1 opens the script with the CWD as c:\Windows\System32\
The remainder open the script with the CWD (as from os.getcwd()) as the current directory of the executed script.

The issue arose when attempting to open a file in the same directory as the script. The issue of how to access a file in the same directory as the script isn't the point of this issue however it is that it is inconsistent across methods that would seem to be identical to the user.

A use case for Open with->python.exe is if say you want your default behavior to be open with an IDE.

The following other issues I found may be relevant.

#22121
#25450

Some digging found this registry entry which may be relevant.
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.py
msg264333 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2016-04-26 22:45
I don't think there's actually a way to fix this - the current working directory is inherited from the current process in the "Open with" case (FWIW, #3 also produces the same result as #1).

I suspect it would need a fix in Windows, which is highly unlikely, or potentially a fix in the launcher to change the current working directory? The latter sounds like a bad idea, but I could be wrong.

In any case, won't be fixed for 2.7, only 3.6 and potentially 3.5.
msg264341 - (view) Author: Tom Middleton (busfault) Date: 2016-04-27 03:09
That is interesting, in my use #3 seems to work. I am not certain if it matters whether the default application is already selected as python.exe or not.
FWIW I'm on Windows 7 64 bit using 2.7.11 release install.

**Full Disclosure** I did muck in the registry to get command line arguments to work correctly.(adding the %* to python.exe commands ala #7936) though that shouldn't have any affect on this issue.
msg264368 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2016-04-27 09:55
Generally the directory that an application needs for its configuration files and data is either the script directory or an %AppData% or %LocalAppData% subdirectory. If the initial working directory matters for some reason (e.g. for writing output files), the parent process either changes to the desired directory beforehand or passes the working directory as a parameter to the CreateProcess or ShellExecuteEx API. It wouldn't make sense for Python to set the working directory to the script directory, because a standard user probably can't even modify that directory. 

> From Explorer:

When running a file, Explorer usually creates the child process with the initial working directory set to that of the target file. However, for "open with" it uses its own working directory, "%SystemRoot%\System32". Actually, the "open with" dialog (not the menu) that lets you select and run an application isn't even Explorer. It's openwith.exe. At least it is in Windows 10 and, IIRC, back to Windows 7.

A workaround is to create a "Run" command in "HKCU\SOFTWARE\Classes\SystemFileAssociations\.py". See the following page for more information about application registration:

https://msdn.microsoft.com/en-us/library/ee872121

For example, the follow .reg file defines a command to run a script using the py.exe launcher. It also adds an "Edit" command that uses DDE to open a script in Visual Studio 2015. 

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\SOFTWARE\Classes\SystemFileAssociations\.py]

[HKEY_CURRENT_USER\SOFTWARE\Classes\SystemFileAssociations\.py\shell]
@="Run"

[HKEY_CURRENT_USER\SOFTWARE\Classes\SystemFileAssociations\.py\shell\Edit]

[HKEY_CURRENT_USER\SOFTWARE\Classes\SystemFileAssociations\.py\shell\Edit\command]
@="\"C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\Common7\\IDE\\devenv.exe\" /dde"

[HKEY_CURRENT_USER\SOFTWARE\Classes\SystemFileAssociations\.py\shell\Edit\ddeexec]
@="Open(\"%1\")"

[HKEY_CURRENT_USER\SOFTWARE\Classes\SystemFileAssociations\.py\shell\Edit\ddeexec\application]
@="VisualStudio.14.0"

[HKEY_CURRENT_USER\SOFTWARE\Classes\SystemFileAssociations\.py\shell\Edit\ddeexec\topic]
@="system"

[HKEY_CURRENT_USER\SOFTWARE\Classes\SystemFileAssociations\.py\shell\Run]

[HKEY_CURRENT_USER\SOFTWARE\Classes\SystemFileAssociations\.py\shell\Run\command]
@="\"C:\\Windows\\py.exe\" \"%1\" %*"

These commands are associated directly with the .py file extension, so they'll always be available, even if no ProgID (e.g. Python.File) is associated with .py scripts. However, the shell will prefer a selected ProgID's "Run" or "Edit" commands, if they exist.

FYI, Explorer's "FileExts\.py" key caches the OpenWithProgids, OpenWithList, and the selected UserChoice for .py files. It shouldn't be modified programmatically, and there's actually a deny ACE on the UserChoice subkey to prevent setting values. One thing you can do, if necessary, is delete the entire key to recompute the file association from the HKCR settings.

If you're manually testing changes by directly modifying the registry (such as deleting the above key), you can call the shell's SHChangeNotify function to refresh file associations, which avoids having to log off and back on. For example:

    import ctypes
    shell32 = ctypes.WinDLL('shell32')

    SHCNE_ASSOCCHANGED = 0x08000000
    shell32.SHChangeNotify.restype = None
    
    shell32.SHChangeNotify(SHCNE_ASSOCCHANGED, 0, 0, 0)

> from the command prompt:

cmd spawns a process using its current working directory (displayed in the prompt, or returned by "cd" without an argument). This is not necessarily the directory of the target file. You can use the /d option of the start command to override this with a specific working directory. For example:

    start "title" /d "C:\Temp" "C:\Temp\somescript.py"
msg264380 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2016-04-27 16:20
Wouldn't be surprised to see differences between Win7 and Win10 - the "current working directory" concept has essentially been deprecated in Windows for a while (at least through the GUI).

Registering a system file association is something we could do for the launcher, but that'd be about it. I'm not sure it'd gain that much. Would be very interesting to collect data on who double clicks .py files vs. running them from another program or a console.
msg264381 - (view) Author: Tom Middleton (busfault) Date: 2016-04-27 17:17
I agree with your assessment Steve. I don't see there being a good fix to this. I also think it would be a bad idea to have the launcher change the current working directory.
Example:
c:\foo\> python d:\scripts\bar.py myfile
(where myfile is in c:\foo\)

A kludge work around in a script is something like this:
import os

if __name__ == "__main__":
    os.chdir(os.path.dirname(__file__))


I also don't think it makes sense to counteract what seems to be a Windows issue in python (or the launcher of which I am less familiar with). I was hoping it was going to be a simple registry setting or something like that which could be handled in the install process.


Eryk, you are correct, it was poor coding that led to the issue. I don't think I was clear in my initial post that I realize that opening a file and assuming the directory of the script is the cwd is not good practice. I was commenting that it is inconsistent between launching methods. I was launching a script that was attempting to write a log file using the logging module with a relative path filename. It was a gotcha because it worked fine on my desktop and it wasn't working when a colleague ran it from "Open with". I've since improved my methodology so the issue is moot for that regard.
However, it is still interesting that there is an inconsistency in Windows in those methods of execution. FWIW I also tested it out with perl and as I expected, got the same results of working directories.

Also please note that I am working in Python 2.7, there is no py launcher as there is in 3. I am not certain what inconsistencies may occur between calling py.exe v. python.exe.
History
Date User Action Args
2022-04-11 14:58:30adminsetgithub: 71053
2021-02-25 10:39:12eryksunsetstatus: open -> closed
resolution: third party
stage: resolved
2016-04-27 17:17:27busfaultsetmessages: + msg264381
2016-04-27 16:20:38steve.dowersetmessages: + msg264380
2016-04-27 09:55:50eryksunsetnosy: + eryksun
messages: + msg264368
2016-04-27 03:09:59busfaultsetmessages: + msg264341
2016-04-26 22:45:36steve.dowersetmessages: + msg264333
versions: + Python 3.5, Python 3.6, - Python 2.7
2016-04-26 16:01:58busfaultcreate