This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Two improvements for the locale aliasing engine
Type: enhancement Stage:
Components: Library (Lib), Windows Versions: Python 3.3
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: lemburg, r.david.murray, ssegvic, vstinner
Priority: normal Keywords:

Created on 2011-09-12 12:41 by ssegvic, last changed 2022-04-11 14:57 by admin.

Messages (3)
msg143902 - (view) Author: Sinisa Segvic (ssegvic) Date: 2011-09-12 12:41

There appears to be some space for improvement
regarding the stable implementation
of the Python locale aliasing engine.

Sometimes, one wishes to be able
to override the default system locale.
For instance, it would be nice that a program
supposed to sort people according to national rules
would be able to run correctly even when the system
does not default to the national locale.

Judging from the Python manuals and
provided the desired national locale is installed,
this should be doable in at least the following two ways:

import locale
ianaLanguageSubtag='en'       # the desired national locale
  (ianaLanguageSubtag, locale.getpreferredencoding())) #(1)
  locale.normalize(ianaLanguageSubtag))                #(2)

For a quicker reference, this is
the relevant part of the manual:
locale.setlocale(category, locale=None)
  If (the locale) is a tuple, it is converted
  to a string using the locale aliasing engine.

The locale aliasing engine binds
the IANA language subtags to POSIX locales.
Its effects can be directly observed
through locale.normalize:
>>> import locale
>>> locale.normalize('hr')
>>> locale.normalize('en')

My first objection concerns the Windows behaviour
of the calls (1) and (2) above.
Both of the two calls *do not work* since Windows
does not recognize strings such as 'en_US.ISO8859-1'.
Instead, Windows provides their own locale nomenclature:
Consequently, the following *works*:

locale.setlocale(locale.LC_ALL, 'English_United States.1252')

IMHO this issue should be fixed, perhaps by providing
an alternate definition of the locale alias dictionary

My second objection concerns the behaviour on Linux,
where the call (1) above always works,
while (2) in some cases might not work.
It happens that the call (2)
requests an outdated 8-bit encoding
although UTF8 has obtained a world-wide acceptance.
The call shall result in a locale error
whenever the desired national locale
is present only in the UTF8 variant.

This might be fixed by changing the encodings
in the locale.locale_alias from 8-bit variants to UTF8.
Note however that the problem could be circumvented
by issuing the call (1), so this would be
less important than the Windows fix proposed above.

Source code references:

comp.lang.python discussion:


msg185922 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2013-04-03 13:12
From msg143902 "Both of the two calls *do not work* since Windows
does not recognize strings such as 'en_US.ISO8859-1'." so how can this be an enhancement?

But anyway #10466 talks about the same thing, could this be closed as a duplicate?
msg185923 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-04-03 13:35
I believe Martin is working on a proposal that will solve the windows issue, but it may be quite some time before that happens.

Yes, the windows issue is essentially a dup, but the linux issue is different.

As for your question, fixing the Windows issue is definitely an enhancement, since it requires mapping (or providing, as Martin proposes) the standard locale names on Windows, which is a service that currently doesn't exist.  The linux issue might be a bug...we sometimes update alias tables in maintenance release.
Date User Action Args
2022-04-11 14:57:21adminsetgithub: 57173
2014-02-03 18:35:17BreamoreBoysetnosy: - BreamoreBoy
2013-04-03 13:35:06r.david.murraysetnosy: + r.david.murray
messages: + msg185923
2013-04-03 13:12:44BreamoreBoysetnosy: + BreamoreBoy
messages: + msg185922
2011-09-17 15:37:59eric.araujosettype: behavior -> enhancement
versions: + Python 3.3, - Python 3.2
2011-09-12 12:46:44vstinnersetnosy: + lemburg, vstinner
2011-09-12 12:41:26ssegviccreate