classification
Title: Deprecate getdefaultlocale(), getlocale() and normalize() functions
Type: Stage:
Components: Library (Lib) Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: lemburg, vstinner
Priority: normal Keywords:

Created on 2021-03-19 13:27 by vstinner, last changed 2021-03-19 14:42 by lemburg.

Messages (2)
msg389086 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-03-19 13:27
I propose to deprecate getdefaultlocale(), getlocale() and normalize() functions since they have multiple issues, and remove them in Python 3.12.


The normalize() function uses the locale.locale_alias dictionary which was copied from the X11 locale database in 2000. It's hard to keep this dictionary up to date and to support all locales of all platforms supported by Python. There are multiple issues on macOS for example.

getdefaultlocale() and getlocale() use heuristics to get an encoding from the locale name. These heuristics are not reliable.

getdefaultlocale() only rely on environment variables. When setlocale() is called, environment variables are not updated, and so the encoding returned by getdefaultlocale() is not the effective LC_CTYPE locale encoding. Example:
https://bugs.python.org/issue43552#msg389069

getlocale() open issues:

* bpo-20088: locale.getlocale() fails if locale name doesn't include encoding 
* bpo-23425: Windows getlocale unix-like with french, german, portuguese, spanish 
* bpo-33934: locale.getlocale() seems wrong when the locale is yet unset (python3 on linux) 
* bpo-38805: locale.getlocale() returns a non RFC1766 language code 
* bpo-43115: locale.getlocale fails if locale is set 

getdefaultlocale() open issue:

* bpo-6981: locale.getdefaultlocale() envvars default code and documentation mismatch 
* bpo-30755: locale.normalize() and getdefaultlocale() convert C.UTF-8 to en_US.UTF-8


Replacements:

* getdefaultlocale()[1] => getpreferredencoding(False) or get_current_locale_encoding(), see bpo-43552
* getlocale(loc) => setlocale(loc) or setlocale(loc, None)
* normalize => no replacement. There is no standard way to normalize a locale name.
msg389096 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2021-03-19 14:42
+1 on getdefaultlocale() as mentioned in https://bugs.python.org/issue43552

However, -1 on getlocale() and normalize().

Those two are needed to access and successfully set the locale on
Linux: the lib C setlocale() is very picky about locale names and
so normalization helps in finding the right form and getting
usable results across platforms.

The issues open for these should be addressed and fixed.
History
Date User Action Args
2021-03-19 14:42:31lemburgsetnosy: + lemburg
messages: + msg389096
2021-03-19 13:27:27vstinnercreate