Author eryksun
Recipients eryksun, paul.moore, steve.dower, tim.golden, xtreak, zach.ware
Date 2019-08-26.20:51:32
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
We get into trouble with test_getsetlocale_issue1813 because normalize() maps "tr_TR" (supported) to "tr_TR.ISO8859-9" (not supported).

    >>> locale.normalize('tr_TR')

We should skip normalize() in Windows. It's based on a POSIX locale_alias mapping that can only cause problems. The work for normalizing locale names in Windows is best handled inline in _build_localename and _parse_localename.

For the old long form, C setlocale always returns the codepage encoding (e.g. "Turkish_Turkey.1254") or "utf8", so that's simple to parse. For BCP 47 locales, the encoding is either "utf8" or "utf-8", or nothing at all. For the latter, there's an implied legacy ANSI encoding. This is used by the CRT wherever we depend on byte strings, such as in time.strftime:


    >>> locale.setlocale(locale.LC_CTYPE, 'en_GB')
    >>> time.strftime("\u0100")


    >>> locale.setlocale(locale.LC_CTYPE, 'en_GB.utf-8')
    >>> time.strftime("\u0100")

(We should switch back to using wcsftime if possible.)

The implicit BCP-47 case can be parsed as `None` -- e.g. ("tr_TR", None). However, it might be useful to support getting the ANSI codepage via GetLocaleInfoEx [1]. A high-level function in locale could internally call _locale.getlocaleinfo(locale_name, LOCALE_IDEFAULTANSICODEPAGE). This would return a string such as "1254". or "0" for a Unicode-only language. 

For _build_localename, we can't simply limit the encoding to UTF-8. We need to support the old long/abbreviated forms (e.g. "trk_TUR", "turkish_Turkey") in addition to the newer BCP 47 locale names. In the old form we have to support the following encodings:

    * codepage encodings, with an optional "cp" prefix that has 
      to be stripped, e.g. ("trk_TUR", "cp1254") -> "trk_TUR.1254"
    * "ACP" in upper case only -- for the ANSI codepage of the 
    * "utf8" (mixed case) and "utf-8" (mixed case)

(The CRT documentation says "OEM" should also be supported, but it's not.)

A locale name can also omit the language in the old form -- e.g. (None, "ACP") or (None, "cp1254"). The CRT uses the current language in this case. This is discouraged because the result may be nonsense.

Date User Action Args
2019-08-26 20:51:32eryksunsetrecipients: + eryksun, paul.moore, tim.golden, zach.ware, steve.dower, xtreak
2019-08-26 20:51:32eryksunsetmessageid: <>
2019-08-26 20:51:32eryksunlinkissue37945 messages
2019-08-26 20:51:32eryksuncreate