classification
Title: Windows Python cannot handle an early PATH entry containing ".." and python.exe
Type: crash Stage: resolved
Components: Windows Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: Ray Donnelly, eryksun, miss-islington, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2017-12-30 18:58 by Ray Donnelly, last changed 2018-05-28 22:14 by steve.dower. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 5756 merged steve.dower, 2018-02-19 16:56
PR 5817 merged miss-islington, 2018-02-22 18:40
PR 5818 merged steve.dower, 2018-02-22 19:27
Messages (8)
msg309242 - (view) Author: Ray Donnelly (Ray Donnelly) * Date: 2017-12-30 18:58
Over on the Anaconda Distribution we received a (private) bug report about a crash when trying to use scons. I thought initially it was due to one of our patches but I tested it out with official CPython and also with WinPython and ran into the same crash.

To reproduce this, from cmd.exe on CPython (here I installed CPython as part of Visual Studio 2017, then updated it to the latest 3.6.4 release):

```
set "PATH=C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\Scripts\..;%PATH%"

python.exe

..

python
Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00000328 (most recent call first):
```

The trigger for this bug is the following batch code in scons.bat:

https://bitbucket.org/scons/scons/src/c0172db149b1a151eeb76910d55c81746bfede05/src/script/scons.bat?at=default&fileviewer=file-view-default#scons.bat-19

My current thinking is that the best fix here is to modify get_progpath()/get_program_full_path() so that it uses PathCchCanonicalizeEx() at https://github.com/python/cpython/blob/9bee329130aae5a13050c08dab9d349b76e66835/PC/getpathp.c#L558-L559
msg309243 - (view) Author: Ray Donnelly (Ray Donnelly) * Date: 2017-12-30 19:06
.. though I will also ask the scons people to change this to use pushd and %CD% instead. Even if you were to make Python capable of handling such bad input, who knows what other programs will fail, and build systems should be extra careful not to mess the environment up like this.
msg309249 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2017-12-30 20:10
Here's a way to trigger this error that's unrelated to the PATH environment variable:

    >>> subprocess.call('python', executable=r'C:\Program Files\Python36\.\python.exe')
    Fatal Python error: Py_Initialize: unable to load the file system codec
    ModuleNotFoundError: No module named 'encodings'
    [...]

Apparently Windows doesn't normalize the process image path if it uses only backslash as the path separator. It normalizes it if at least one backslash is replaced with a slash.
msg309491 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-01-05 01:44
I agree that explicitly normalizing in PC/getpathp.c is the correct approach.

As far as I'm aware, GetModuleFileNameW(NULL) can never fail other than a buffer that is too small, so that whole function can probably be simplified to abort if it happens and just remove the PATH search.

Also, we should use a similar process for PathCchCanonicalizeEx as we already use for PathCchCombineExW in this file to avoid failing on earlier OS versions.
msg309495 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2018-01-05 09:25
For extra measure, you may want to normalize `prefix` prior to calculating its length n in gotlandmark(). Then it would be reliable to truncate it via `prefix[n] = '\0'` after joining with `landmark`. Or at least add a comment there or in the calling function, search_for_prefix(), that gotlandmark assumes the current value of `prefix` is a canonical, normalized path.
msg312579 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-02-22 18:39
New changeset 48e8c82fc63d2ddcddce8aa637a892839b551619 by Steve Dower in branch 'master':
bpo-32457: Improves handling of denormalized executable path when launching Python (GH-5756)
https://github.com/python/cpython/commit/48e8c82fc63d2ddcddce8aa637a892839b551619
msg312587 - (view) Author: miss-islington (miss-islington) Date: 2018-02-22 19:15
New changeset e5a9b3574c89a070fbc43caf541759f504e3d492 by Miss Islington (bot) in branch '3.7':
bpo-32457: Improves handling of denormalized executable path when launching Python (GH-5756)
https://github.com/python/cpython/commit/e5a9b3574c89a070fbc43caf541759f504e3d492
msg312595 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-02-22 20:33
New changeset 1d3c518c5ecbd78478738f068f4f035f81f035f9 by Steve Dower in branch '3.6':
bpo-32457: Improves handling of denormalized executable path when launching Python (GH-5756) (#5818)
https://github.com/python/cpython/commit/1d3c518c5ecbd78478738f068f4f035f81f035f9
History
Date User Action Args
2018-05-28 22:14:20steve.dowersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-02-22 20:33:18steve.dowersetmessages: + msg312595
2018-02-22 19:27:45steve.dowersetpull_requests: + pull_request5594
2018-02-22 19:15:19miss-islingtonsetnosy: + miss-islington
messages: + msg312587
2018-02-22 18:40:22miss-islingtonsetpull_requests: + pull_request5592
2018-02-22 18:39:28steve.dowersetmessages: + msg312579
2018-02-19 16:56:21steve.dowersetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request5534
2018-02-19 16:55:31steve.dowersetassignee: steve.dower
2018-01-05 09:25:46eryksunsetmessages: + msg309495
2018-01-05 01:44:58steve.dowersetstage: needs patch
versions: + Python 3.7, Python 3.8
2018-01-05 01:44:42steve.dowersetmessages: + msg309491
2017-12-30 20:10:13eryksunsetnosy: + eryksun
messages: + msg309249
2017-12-30 19:06:58Ray Donnellysetmessages: + msg309243
2017-12-30 18:58:32Ray Donnellycreate