classification
Title: Windows Store installer should warn user about MAX_PATH
Type: enhancement Stage:
Components: Windows Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Jonas Binding, Marcus.Smith, dstufft, eryksun, ncoghlan, paul.moore, pradyunsg, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2019-08-05 22:02 by Jonas Binding, last changed 2019-09-11 10:51 by steve.dower.

Pull Requests
URL Status Linked Edit
PR 15914 open steve.dower, 2019-09-11 10:49
Messages (5)
msg349079 - (view) Author: Jonas Binding (Jonas Binding) Date: 2019-08-05 22:02
The "Windows Store" installer for Python has a seemingly low entry barrier, causing people to install without reading something like https://docs.python.org/3.7/using/windows.html. 

However, due to the really long path it uses for Python (e.g. C:\Users\USERNAME\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\), the chances of running into issues with paths longer than 260 characters are really high. 

For example installing pytorch already fails due to this limitation, see https://github.com/pytorch/pytorch/issues/23823 and https://www.reddit.com/r/pytorch/comments/c6cllq/issue_installing_pytorch/

Therefore, the Windows Store installer should offer to change the registry key to enable longer paths, or at least show a message to this effect.
msg349083 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-08-05 22:24
Short of adding a popup into Python itself the first time you run it (and would that include if you run "pip" first?), we don't have any ability to extend the installer.

We could reduce the "local-packages/Python37/site-packages" part of the path by implementing an alternative way (in both Python and pip, presumably) to specify the user's site packages. Right now, the best I can do is reduce "local-packages" down to a single character (in PC/python_uwp.cpp ) - the rest is forcibly added by the shared part of the runtime.

Since pip is likely to be the first place users hit this, it might be easiest to start by adding a more descriptive error message. Copying from the other bug:

pip3 install https://download.pytorch.org/whl/cpu/torch-1.1.0-cp37-cp37m-win_amd64.whl
fails with an error message:

ERROR: Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: 'C:\\Users\\[username]\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python37\\site-packages\\caffe2\\python\\serialized_test\\data\\operator_test\\collect_and_distribute_fpn_rpn_proposals_op_test.test_collect_and_dist.zip'

First it seems the error is being raised incorrectly - winerror 2 is for file not found, but it's being passed as errno 2. But otherwise it should be possible to detect this case, see that the path is too long on Windows and point users at https://docs.python.org/3/using/windows.html#removing-the-max-path-limitation
msg349086 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2019-08-05 22:44
> First it seems the error is being raised incorrectly - winerror 2 is 
> for file not found, but it's being passed as errno 2. 

I think this only happens with open(). The _io module could use the CRT's _doserrno value to call PyErr_SetExcFromWindowsErrWithFilenameObject. We can rely on _doserrno if _wopen fails.

Otherwise this is the expected mapping from Windows ERROR_PATH_NOT_FOUND (3) or ERROR_FILENAME_EXCED_RANGE (206) to POSIX ENOENT (2). The Windows error in the case of path that's too long is not ERROR_FILE_NOT_FOUND (2).
msg349087 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2019-08-05 22:48
> I think this only happens with open(). 

Well, and everything else that calls a CRT function and relies on errno, such as C read() and write(). Though we'd have to look through on a case-by-case basis to ensure that _doserrno is valid in each case, i.e. that errno was set based on a Windows error code.
msg351811 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-09-11 10:51
I don't know that it's a great idea, but I did PR 15914 just to see how an additional message on OSError would look.
History
Date User Action Args
2019-09-11 10:51:04steve.dowersetmessages: + msg351811
stage: patch review ->
2019-09-11 10:49:55steve.dowersetkeywords: + patch
stage: patch review
pull_requests: + pull_request15555
2019-08-05 22:48:55eryksunsetmessages: + msg349087
2019-08-05 22:44:32eryksunsetnosy: + eryksun
messages: + msg349086
2019-08-05 22:24:23steve.dowersetnosy: + ncoghlan, dstufft, pradyunsg, Marcus.Smith
messages: + msg349083
2019-08-05 22:02:48Jonas Bindingcreate