classification
Title: Bug in C locale coercion with PYTHONCOERCECLOCALE=1
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: vstinner
Priority: normal Keywords: patch

Created on 2018-11-28 10:42 by vstinner, last changed 2018-11-30 11:20 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 10806 merged vstinner, 2018-11-29 23:25
PR 10813 merged vstinner, 2018-11-30 10:45
Messages (4)
msg330590 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-28 10:42
PYTHONCOERCECLOCALE=1 should not force C locale coercion, Python should still check if the LC_CTYPE locale is "C".

Bug:

$ ./python -c 'import locale; print(locale.setlocale(locale.LC_CTYPE, None))'
fr_FR.UTF-8
$ PYTHONCOERCECLOCALE=1 ./python -c 'import locale; print(locale.setlocale(locale.LC_CTYPE, None))'
C.UTF-8

It should be fr_FR.UTF-8 as well in the second example :-( It seems to be a regression that I introduced in the middle of my refactoring on the Python initialization.

The bug is around:

static void
config_init_locale(_PyCoreConfig *config)
{
    if (config->coerce_c_locale < 0) {
        /* The C locale enables the C locale coercion (PEP 538) */
        if (_Py_LegacyLocaleDetected()) {
            config->coerce_c_locale = 1;
        }
    }
    ...
}

The 3.7 branch and the 3.7.0 release are affected :-(

$ ./python -V
Python 3.7.0
$ PYTHONCOERCECLOCALE=1 ./python -c 'import locale; print(locale.setlocale(locale.LC_CTYPE, None))'
C.UTF-8
msg330768 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-30 10:34
New changeset 55e498058faf8c97840556f6d791c2c392732dc3 by Victor Stinner in branch 'master':
bpo-35336: Fix PYTHONCOERCECLOCALE=1 (GH-10806)
https://github.com/python/cpython/commit/55e498058faf8c97840556f6d791c2c392732dc3
msg330771 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-30 11:18
> PYTHONCOERCECLOCALE=1 should not force C locale coercion, Python should still check if the LC_CTYPE locale is "C".

Reference:

https://docs.python.org/dev/using/cmdline.html#envvar-PYTHONCOERCECLOCALE

"If (...) the current locale reported for the LC_CTYPE category is either the default C locale, or else the explicitly ASCII-based POSIX locale, (...)"
msg330772 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-11-30 11:19
New changeset df738d56fe798b3586ed71775df25bf127789cf6 by Victor Stinner in branch '3.7':
bpo-35336: Fix PYTHONCOERCECLOCALE=1 (GH-10806) (GH-10813)
https://github.com/python/cpython/commit/df738d56fe798b3586ed71775df25bf127789cf6
History
Date User Action Args
2018-11-30 11:20:02vstinnersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-11-30 11:19:50vstinnersetmessages: + msg330772
2018-11-30 11:18:31vstinnersetmessages: + msg330771
2018-11-30 10:45:27vstinnersetpull_requests: + pull_request10058
2018-11-30 10:34:50vstinnersetmessages: + msg330768
2018-11-29 23:25:28vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request10052
2018-11-28 10:42:16vstinnercreate