This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: [venv / PC/launcher] issue with a space in the installed python path
Type: behavior Stage: needs patch
Components: Windows Versions: Python 3.11, Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, hokiedsp, paul.moore, steve.dower, tim.golden, zach.ware
Priority: critical Keywords:

Created on 2022-02-08 22:48 by hokiedsp, last changed 2022-04-11 14:59 by admin.

Messages (6)
msg412874 - (view) Author: Kesh Ikuma (hokiedsp) Date: 2022-02-08 22:48
After months of proper operation, my per-user Python install started to error out when I attempt `python -m venv .venv` with 

"Error: Command '['C:\\Users\\kesh\\test\\.venv\\Scripts\\python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 101."

Following the StackOverflow solution, I reinstalled Python for all users and it was working OK. I recently looked into it deeper and found the root issue in the function PC/launcher.c/run_child(). The path to the "...\Python\Python310\python.exe" contains a space, and the CreateProcessW() call on Line 811 is passing the path without quoting the path, causing the process creation to fail.

I fixed my issue by using the Windows short path convention on my path env. variable, but there must be a more permanent fix possible.

Here is the link to my question and self-answering to the problem:

https://stackoverflow.com/questions/71039131/troubleshooting-the-windows-venv-error-101
msg412883 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2022-02-09 00:29
run_child() expects `cmdline` to be correctly quoted, and normally it is.

I can't reproduce this problem with Python 3.10.2. I created a user account with a space in the account name, logged on, and installed 3.10.2 for the current user, with the option enabled to add Python to PATH. Next I opened a command prompt in the user profile directory and created a virtual environment via `python.exe -m venv .venv`. Running ".venv\Scripts\python.exe -X utf8" worked. The command line used by the venv "python.exe" launcher was properly quoted and thus properly parsed in the original list of command-line arguments, sys.orig_argv.
msg412885 - (view) Author: Kesh Ikuma (hokiedsp) Date: 2022-02-09 00:42
@eryksun - I knew the reproducibility is the issue with this bug. On the same PC I've been having a problem with, I created another dummy account with a space in its username, and it worked flawlessly... So, it's something I've done to my account which triggered this behavior. 

Is there anything that I can try and report on my machine? I'd be happy to do so
msg412889 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2022-02-09 04:29
I checked the source code in PC/launcher.c process(). It turns out that `executable` is not getting quoted in the venv launcher case.

CreateProcessW() tries to get around this. If the command isn't quoted, it has a loop that consumes up to a space (or tab) and checks for an existing file (not a directory). If it finds a file, it rewrites the command line to quote the path of the file.

My test happened to work. But it's simple enough to create an example that fails. For example, as an elevated admin, create a file named "C:\Program". Now the venv launcher won't be able to execute a base interpreter that's installed in "C:\Program Files":

    C:\Temp>echo >C:\Program
    
    C:\Temp>"C:\Program Files\Python310\python.exe" -m venv env
    Error: Command '['C:\\Temp\\env\\Scripts\\python.exe', '-Im', 
    'ensurepip', '--upgrade', '--default-pip']' returned non-zero 
    exit status 101.
msg412894 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2022-02-09 06:28
The venv launcher can quote the executable path either always or only when it contains spaces. The following is a suggestion for implementing the latter.

Before allocating `executable`, use memchr() (include <memory.h>) to search the UTF-8 source for a space. If found, increment the character count by two. After allocating `executable`, add the initial quote character, and increment the pointer.

        BOOL add_quotes = FALSE;
        if (memchr(start, ' ', (size_t)len) != NULL) {
            add_quotes = TRUE;
            cch += 2;
        }

        executable = (wchar_t *)malloc(cch * sizeof(wchar_t));
        if (executable == NULL) {
            error(RC_NO_MEMORY, L"A memory allocation failed");
        }
        
        if (add_quotes) {
            *executable++ = L'\"';
        }

Later, after checking the existence via GetFileAttributesW(), add the trailing quote and null, and decrement the pointer:

        if (GetFileAttributesW(executable) == INVALID_FILE_ATTRIBUTES) {
            error(RC_NO_PYTHON, L"No Python at '%ls'", executable);
        }

        if (add_quotes) {
            size_t n = wcslen(executable);
            executable[n] = L'\"';
            executable[n + 1] = L'\0';
            executable--;
        }
msg412918 - (view) Author: Kesh Ikuma (hokiedsp) Date: 2022-02-09 16:01
> a file named "C:\Program". Now the venv launcher won't be able to execute

This is exactly what happened on my PC, and the behavior was triggered by Microsoft Visual C++ 2015-2022 Redistributable installer. The installer left a log file "C:\Users\Kesh" with my account "C:\Users\Kesh Ikuma".

Thank you for quickly addressing the issue
History
Date User Action Args
2022-04-11 14:59:55adminsetgithub: 90844
2022-02-09 16:01:34hokiedspsetmessages: + msg412918
2022-02-09 06:28:54eryksunsetmessages: + msg412894
2022-02-09 04:29:51eryksunsetpriority: normal -> critical

stage: needs patch
messages: + msg412889
versions: + Python 3.11
2022-02-09 00:42:11hokiedspsetmessages: + msg412885
2022-02-09 00:29:18eryksunsetnosy: + eryksun
messages: + msg412883
2022-02-08 22:48:39hokiedspcreate