Issue 42568: Python can't run .pyc files with non-ASCII path on Windows

classification

Title:	Python can't run .pyc files with non-ASCII path on Windows
Type:	behavior	Stage:	resolved
Components:	Interpreter Core, Windows	Versions:	Python 3.10, Python 3.9, Python 3.8

process

Status:	closed	Resolution:	duplicate
Dependencies:		Superseder:	Python 3.6 cannot reopen .pyc file with non-ASCII path View: 32381
Assigned To:		Nosy List:	izbyshev, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
Priority:	normal	Keywords:

Created on 2020-12-04 15:03 by izbyshev, last changed 2020-12-04 15:24 by eryksun. This issue is now closed.

Messages (1)
msg382490 - (view)	Author: Alexey Izbyshev (izbyshev) *	Date: 2020-12-04 15:03
> python тест.pyc python: Can't reopen .pyc file The issue is caused by _Py_fopen() being used as though it can deal with paths encoded in FS-default encoding (UTF-8 by default on Windows), but in fact it's just a simple wrapper around fopen() from the C runtime, so it uses the current ANSI code page, breaking if PYTHONLEGACYWINDOWSFSENCODING is not enabled. I could find only two callers if _Py_fopen() on Windows: * https://github.com/python/cpython/blob/db68544122f5/Python/pythonrun.c#L380 (which caused this issue) * https://github.com/python/cpython/blob/db68544122f5/Python/errors.c#L1708 PyErr_ProgramText() doesn't seem to be called in CPython, but https://github.com/python/cpython/blob/db68544122f5/Include/pyerrors.h#L243 claims that filename is "decoded from the filesystem encoding", which doesn't match the code.

msg382490 - (view)

Author: Alexey Izbyshev (izbyshev) * (Python triager)

Date: 2020-12-04 15:03

> python тест.pyc
python: Can't reopen .pyc file

The issue is caused by _Py_fopen() being used as though it can deal with paths encoded in FS-default encoding (UTF-8 by default on Windows), but in fact it's just a simple wrapper around fopen() from the C runtime, so it uses the current ANSI code page, breaking if PYTHONLEGACYWINDOWSFSENCODING is not enabled.

I could find only two callers if _Py_fopen() on Windows:

* https://github.com/python/cpython/blob/db68544122f5/Python/pythonrun.c#L380 (which caused this issue)

* https://github.com/python/cpython/blob/db68544122f5/Python/errors.c#L1708

PyErr_ProgramText() doesn't seem to be called in CPython, but https://github.com/python/cpython/blob/db68544122f5/Include/pyerrors.h#L243 claims that filename is "decoded from the filesystem encoding", which doesn't match the code.

History
Date	User	Action	Args
2020-12-04 15:24:00	eryksun	set	status: open -> closed superseder: Python 3.6 cannot reopen .pyc file with non-ASCII path resolution: duplicate stage: resolved
2020-12-04 15:03:14	izbyshev	create