This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: PYTHONCOERCECLOCALE=0 ignored
Type: Stage: resolved
Components: Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: lazka, vstinner
Priority: normal Keywords:

Created on 2019-11-02 13:14 by lazka, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (3)
msg355868 - (view) Author: Christoph Reiter (lazka) * Date: 2019-11-02 13:14
Python 3.7.5rc1 and 3.8.0 on Ubuntu 19.10

$ LC_CTYPE=C PYTHONCOERCECLOCALE=warn python3 -c "import sys; print(sys.getfilesystemencoding())"
Python detected LC_CTYPE=C: LC_CTYPE coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOCALE=0 to disable this locale coercion behavior).
utf-8

$ LC_CTYPE=C PYTHONCOERCECLOCALE=0 python3 -c "import sys; print(sys.getfilesystemencoding())"   
utf-8

The warning states that passing PYTHONCOERCECLOCALE=0 disables the coercion, but it doesn't change anything in 3.7 and 3.8.

What am I missing?
msg355919 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-11-04 01:33
You should not use sys.getfilesystemencoding() nor locale.getpreferredencofing() to check if the C locale has been coerced, but read the LC_CTYPE locale instead:

# C locale coerced
$ LC_CTYPE=C python3 -c "import locale; print(locale.setlocale(locale.LC_CTYPE, ''))"
C.UTF-8

# C locale not coerced: remains "C"
$ LC_CTYPE=C PYTHONCOERCECLOCALE=0 python3 -c "import locale; print(locale.setlocale(locale.LC_CTYPE, ''))"
C

--

$ LC_CTYPE=C PYTHONCOERCECLOCALE=0 python3 -c "import sys; print(sys.getfilesystemencoding())"   
utf-8

In this example, the UTF-8 encoding is used because of the UTF-8 Mode:

$ LC_CTYPE=C PYTHONCOERCECLOCALE=0 python3 -X utf8=1 -c "import sys; print(sys.getfilesystemencoding(), sys.flags.utf8_mode)"
utf-8 1

$ LC_CTYPE=C PYTHONCOERCECLOCALE=0 python3 -X utf8=0 -c "import sys; print(sys.getfilesystemencoding(), sys.flags.utf8_mode)"
ascii 0

The relationship between the PEP 538 and the PEP 540 is non obvious:
https://www.python.org/dev/peps/pep-0540/#relationship-with-the-locale-coercion-pep-538

I guess that you would like: PYTHONUTF8=0 env var or -X utf8=0 command line option.

--

I don't think that this issue is a bug, and so I suggest to close it as "not a bug".
msg355927 - (view) Author: Christoph Reiter (lazka) * Date: 2019-11-04 07:18
Thanks! I didn't quite see the relation between PEP 538 and PEP 540 after only reading the former.

Anyone else finding this: See "Relationship with the locale coercion" in https://www.python.org/dev/peps/pep-0540/ for the details.
History
Date User Action Args
2022-04-11 14:59:22adminsetgithub: 82848
2019-11-04 07:18:40lazkasetstatus: open -> closed
resolution: not a bug
messages: + msg355927

stage: resolved
2019-11-04 01:33:18vstinnersetmessages: + msg355919
2019-11-02 14:10:21xtreaksetnosy: + vstinner
2019-11-02 13:14:24lazkacreate