This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Windows 11, VENV not working with case sensitive windows paths
Type: crash Stage:
Components: Windows Versions: Python 3.11
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: darrel.opry, eryksun, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2022-03-07 18:28 by darrel.opry, last changed 2022-04-11 14:59 by admin.

Messages (6)
msg414691 - (view) Author: Darrel O'Pry (darrel.opry) Date: 2022-03-07 18:28
I created a virtual env on windows 11. 
When I run pip install -r "..."
I get the error 
C:\Users\dopry\src\Client\\django> pip install -r .\requirements_to_freeze.txt
Traceback (most recent call last):
  File "C:\Users\dopry\AppData\Local\Programs\Python\Python310\lib\", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\dopry\AppData\Local\Programs\Python\Python310\lib\", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\dopry\src\Client\\django\venv310\Scripts\pip.exe\", line 4, in <module>
ModuleNotFoundError: No module named 'pip'

When I run get command to verify that the venv is activated I get

(venv310) PS C:\Users\dopry\src\Client\\django> Get-Command python

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Application     python.exe                                         3.10.21... C:\Users\dopry\src\Client\\django\venv310\Scripts\python.exe

When I update the `venv310\pyenv.cfg` and set `include-system-site-packages = true` to true, it was defaulted to false, it seems that I can pip install my requirements successfully. 

When I try to start my app though, the requirements are not found.
msg415174 - (view) Author: Darrel O'Pry (darrel.opry) Date: 2022-03-14 18:01
I've done some additional troubleshooting today. I have case sensitivity enabled in my git checkouts where I am creating the virtual env. I do this for a more consistent cross-platform experience for managing code with team members also using linux and macos. I have had issues in the past with case changes in filenames causing issue on projects and enabling case sensitivity has eliminate these issues when working with my counterparts on other platforms. 

I've tested the following with both powershell and bash. 

mkdir test-venv-case-sensitivity
cd test-venv-case-sensitivity
fsutil.exe file setCaseSensitiveInfo . enable
python -m venv ./venv
pip # you should get the error "No module named 'pip'
cd ./venv/
fsutil.exe file setCaseSensitiveInfo . disable
pip # the command now works

This wasn't an issue prior to upgrading to Windows 11, so something may have changed in the handling of the case sensitivity flag on Windows. 

I have a venv created on windows 10, prior to my windows 11 upgrade that does work. 

I didn't encounter the issue until setting up an installation of Python 3.9.10 to support another project, then trying to create a new 3.10.2 venv. 

I can work around the issue by specifically disabling case sentitivity in my venv folders, but it would be nice if venvs worked out of the box with case sensitive filesytems on windows. 

It seems like some code is making assumptions about case sensitivity on windows. 

I found that if I disable case sensitivity in the venv folder, the venv starts working again.
msg415192 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2022-03-14 21:17
As a quick (wild) guess, is it expecting the "Lib" directory to be lowercase "lib"?

Could you try renaming that directory in your venv and see if it changes anything?
msg415195 - (view) Author: Darrel O'Pry (darrel.opry) Date: 2022-03-14 21:29
renaming Lib to lib seems to also resolve the problem...
msg415201 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2022-03-14 22:19
Okay, so that means there's some code somewhere that has a lowercase "lib".

If you change it back to "Lib", can you do "python -m pip"? If that works, but a direct "pip" does not, it'll be an issue in pip (or a dependency), as those executables are generated by them.

There are likely many places where we rely on case-insensitivity throughout the codebase though, and certainly a number of places where we unconditionally casefold on Windows before doing comparisons. It's going to take a decent amount of time to track these down, and may not always be feasible, but if we can start enumerating them then it's worth making the fixes.
msg415211 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2022-03-14 23:57
In 3.10, you should be able to work around the problem for the venv site-packages directory by setting the environment variable "PYTHONPLATLIBDIR" to "Lib". This sets sys.platlibdir, which the site module uses to create the site-packages directory. The default value is "lib", which isn't properly capitalized.

In 3.11, sys.platlibdir defaults to "DLLs" in Windows, and the site module ignores this attribute. However, the site module does hard code the properly capitalized value "Lib".

In POSIX, sys.platlibdir (e.g. "lib" or "lib64") is a common base directory for the standard library and its extension modules. It isn't just the directory for extension modules, as might be implied by changing the value to "DLLs" in Windows. The "DLLs" directory in Windows Python is split off from "Lib" (for some legacy reason, I suppose), so Windows would need separate sys.platlibdir ("Lib") and sys.platextdir ("DLLs") values, if that mattered, which it doesn't since the names are fixed.
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91106
2022-03-14 23:57:16eryksunsetnosy: + eryksun
messages: + msg415211
2022-03-14 22:19:11steve.dowersetmessages: + msg415201
versions: + Python 3.11, - Python 3.9, Python 3.10
2022-03-14 21:29:52darrel.oprysetmessages: + msg415195
2022-03-14 21:17:31steve.dowersetmessages: + msg415192
2022-03-14 18:01:05darrel.oprysetmessages: + msg415174
title: Windows 11 venv -> Windows 11, VENV not working with case sensitive windows paths
2022-03-07 18:28:51darrel.oprycreate