This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: venv fails when called from within long path on Windows
Type: Stage:
Components: Windows Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, paul.moore, schwaerz, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2021-07-05 15:51 by schwaerz, last changed 2022-04-11 14:59 by admin.

Messages (5)
msg397000 - (view) Author: (schwaerz) Date: 2021-07-05 15:51
When trying to create a venv from within a long path name in Windows 10 (long paths enabled in the registry), python crashes. Used python 3.9.6.

When reducing the path length by one character, it starts working.
When disabling the long path support in Windows, I get an error message about paths being too long.

btw.: I can see same / similar behavior when using virtualenv/pipenv/poetry (did not try any other venv-like tooling...)

For the instructions please check the following snippet:

d:\slave\workspace\1--folder-with-232-character-absolute-path-length----------------------------------------------------------------------------------------------------------------------------------------------------------------232>python -m venv .venv
Error: Command '['d:\\slave\\workspace\\1--folder-with-232-character-absolute-path-length----------------------------------------------------------------------------------------------------------------------------------------------------------------232\\.venv\\Scripts\\python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 3221226505.

%Errorlevel% sometimes was 1 and sometimes was -1073740791 (0xc0000409). Unfortunately I cannot recalled what I had to do to get the latter one... I tried with python 3.8.0, too. Same behavior.
msg397010 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-07-05 17:08
The secure CRT string functions such as wcscpy_s() and wcscat_s() invoke the invalid parameter handler if the destination string is too small. This defaults to a fastfail that terminates with the status code 0xC0000409, subcode 5 (FAST_FAIL_INVALID_ARG).

We're using buffers sized MAXPATHLEN+1 in PC/getpathp.c, which supports 256 characters plus a terminating null. We could increase the buffer size, but bear in mind that Windows itself doesn't completely support running applications from a long path. So you may still hit road blocks.
msg397011 - (view) Author: (schwaerz) Date: 2021-07-05 17:15
I see.

Reading https://docs.python.domainunion.de/3/using/windows.html#removing-the-max-path-limitation I thought that the long paths on Win10 are supposed to be supported in general.

Is there any blocker in particular which I should keep in mind?
msg397014 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-07-05 18:07
It enables it for accessing from within Python, but doesn't enable Python itself to be installed into a long path. And because a venv is essentially an installation, they're included in that. (Possibly %PYTHONPATH% entries are too.)

I haven't checked how widely MAXPATHLEN is used, but it may be safe enough to just raise it. However, I think that's going to have a memory impact, because the paths used for initialization are kept around for the whole length of the application now. It might be okay though, not sure how many paths are kept.

I'm still keen to see the whole getpath implementation replaced with Python code (with suitably injected native helpers), which would make it much easier for us to use dynamically allocated strings of any length. The current implementation is a way off being able to handle this.
msg397015 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-07-05 18:08
Oh, and there's no harm in adding a note to that doc section explicitly pointing out that it doesn't enable CPython itself to be installed to a long path, and it may not enable packages to be imported from long paths in all scenarios (as it's very feasible that some 3rd party DLLs won't handle long paths).
History
Date User Action Args
2022-04-11 14:59:47adminsetgithub: 88733
2021-07-05 18:08:39steve.dowersetmessages: + msg397015
2021-07-05 18:07:07steve.dowersetmessages: + msg397014
versions: + Python 3.11, - Python 3.8, Python 3.9
2021-07-05 17:15:18schwaerzsetmessages: + msg397011
2021-07-05 17:08:37eryksunsetnosy: + eryksun
messages: + msg397010
2021-07-05 15:51:01schwaerzcreate