New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ensurepip/venv broken on Windows if path includes unicode #69944
Comments
One of my students installed Python 3.5 on Windows 10 to default location where user name "Łukasz" contains unicode. Now "-m venv" and "-m ensurepip" do not work: C:\Users\Łukasz>C:\Users\Łukasz\AppData\Local\Programs\Python\Python35-32\python.exe -m venv workshops C:\Users\Łukasz>C:\Users\Łukasz\AppData\Local\Programs\Python\Python35-32\python.exe -m ensurepip
Traceback (most recent call last):
File "C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 170, in _run_module_as_main
"__main__", mod_spec)
File "C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\ensurepip\__main__.py", line 4, in <module>
ensurepip._main()
File "C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\ensurepip\__init__.py", line 209, in _main
default_pip=args.default_pip,
File "C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\ensurepip\__init__.py", line 116, in bootstrap
_run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
File "C:\Users\\u0141ukasz\AppData\Local\Programs\Python\Python35-32\lib\ensurepip\__init__.py", line 40, in _run_pip
import pip
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 954, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 896, in _find_spec
File "<frozen importlib._bootstrap_external>", line 1136, in find_spec
File "<frozen importlib._bootstrap_external>", line 1112, in _get_spec
File "<frozen importlib._bootstrap_external>", line 1093, in _legacy_get_spec
File "<frozen importlib._bootstrap>", line 444, in spec_from_loader
File "<frozen importlib._bootstrap_external>", line 530, in spec_from_file_location
UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: invalid character |
The problem is that the compile_source function in Modules/zipimport.c calls PyUnicode_EncodeFSDefault to get an encoded string to pass as the filename argument of Py_CompileString. On Windows this uses the ANSI codepage (i.e. 'mbcs'). Apparently your system's ANSI codepage doesn't map the "Ł" character. I reproduced the problem more simply by copying pip-7.1.2-py2.py3-none-any.whl to a subdirectory named "Łukasz"; adding the wheel path to sys.path; and attempting to execute "import pip". One solution is to replace Py_CompileString with Py_CompileStringObject. This way compile_source doesn't have to worry about encoding its pathname argument. A minimal patch is attached, but it needs a test. |
As a test case for handling non-ASCII characters in the name of the zipfile itself, I believe it should be sufficient to add 'Ł' to TEMP_DIR and TEMP_ZIP in https://hg.python.org/cpython/file/default/Lib/test/test_zipimport.py |
New changeset 663a62bcf9c9 by Steve Dower in branch '3.5': New changeset ead30e7262d5 by Steve Dower in branch 'default': |
Applied Eryk's patch and updated the test to repro it (though it wouldn't repro on 3.6 with PEP-529 applied, but it definitely did on 3.5). |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: