Author eryksun
Recipients eryksun, paul.moore, steve.dower, tim.golden, vinay.sajip, zach.ware
Date 2019-01-24.06:43:33
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1548312213.85.0.779714193672.issue35811@roundup.psfhosted.org>
In-reply-to
Content
It's worth noting the consequences of changing the application directory (i.e. the "%__APPDIR__%" dynamic variable from GetEnvironmentVariableW).
This is the first directory checked in most cases when Windows searches for a file, including the CreateProcess[AsUser]W EXE search path, the default LoadLibrary[Ex]W DLL search path, and the default SearchPathW generic search path. In NT 6.2+, these search paths respectively come from the runtime library functions RtlGetExePath, LdrGetDllPath, and RtlGetSearchPath. The search itself is implemented by RtlDosSearchPath_Ustr, except LoadLibraryExW uses it only when loading a DLL as a data file. (Loading as a module is implemented by LdrLoadDll, which uses a private search function, LdrpSearchPath; resolves API-set names; and reserves known system DLLs.) This search supports file redirection [1], but it's unlikely that Python scripts are creating their own activation contexts.

In practice maybe no scripts will be affected by this change. I'm just noting potential problems.

Changing the application directory will break scripts that depend on the environment's "Scripts" directory being in one of the above search paths. It will also break scripts that depend on it being the first directory searched in order to shadow system files that aren't otherwise reserved.

We can lessen the impact of this change by prepending the "Scripts" directory to PATH. However, some directories will unavoidably have precedence over it now. Specifically, for the CreateProcessW EXE search, it's any of the following directories:

    * "%__APPDIR__%"
        This is now the installed location of python.exe.
    * "%__CD__%"
        The current directory is searched if a name has a 
        backslash in it, and also if the environment variable
        NoDefaultCurrentDirectoryInExePath is not defined.
    * "%SystemRoot%\System32" (GetSystemDirectory)
    * "%SystemRoot%\System"
    * "%SystemRoot%" (GetWindowsDirectory)

Also, applications that call SetDefaultDllDirectories get a secure DLL search path that can only reference "%__APPDIR__%", "%SystemRoot%\System32", and directories added via AddDllDirectory or SetDllDirectory. They no longer use "%__CD__%", "%SystemRoot%\System", "%SystemRoot%", or PATH. An affected script will have to manually add the "Scripts" directory to the DLL search path via AddDllDirectory or SetDllDirectory. 

Scripts that wish to restrict searches to PATH should use shutil.which(). It also works more predictably for relative filenames. In particular, a relative filename that has a path separator in it (e.g. r"eggs\spam.txt") is only searched relative to the process current directory. In contrast, searches based on RtlDosSearchPath_Ustr handle all relative filenames, whether or not they contain a path separator, by trying every directory in the search path (e.g. it will look for r"%__APPDIR__%\eggs\spam.txt", and so on).


[1] If enabled by the call flags, searching for a relative filename
    can be redirected by an activation context or (in the absence 
    of a manifest) an "<exename>.local" directory in the application 
    directory, via RtlDosApplyFileIsolationRedirection_Ustr. 
    CreateProcessW does not enable redirection. SearchPathW enables 
    it for the default search path. LoadLibraryExW / LdrLoadDll needs 
    to redirect even fully-qualified filenames, so it directly calls
    RtlDosApplyFileIsolationRedirection_Ustr.
History
Date User Action Args
2019-01-24 06:43:34eryksunsetrecipients: + eryksun, paul.moore, vinay.sajip, tim.golden, zach.ware, steve.dower
2019-01-24 06:43:33eryksunsetmessageid: <1548312213.85.0.779714193672.issue35811@roundup.psfhosted.org>
2019-01-24 06:43:33eryksunlinkissue35811 messages
2019-01-24 06:43:33eryksuncreate