classification
Title: Add UTF-8 locale aliases
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.5, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: ezio.melotti, lemburg, loewis, python-dev, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2013-12-26 23:29 by serhiy.storchaka, last changed 2014-10-02 07:23 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
locale_utf8_aliases.patch serhiy.storchaka, 2013-12-26 23:29 review
Messages (5)
msg206974 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-12-26 23:29
The makelocalealias.py ignores UTF-8 mapping. Expected that this encoding is available for all locales. After enabling UTF-8 mapping in makelocalealias.py most aliases are optimized out except following:

+    'be_bg.utf8':                           'bg_BG.UTF-8',
+    'c.utf8':                               'en_US.UTF-8',
+    'en_dl.utf8':                           'en_DL.UTF-8',
+    'en_zw.utf8':                           'en_ZS.UTF-8',
+    'ks_in@devanagari.utf8':                'ks_IN.UTF-8@devanagari',
+    'pa_pk.utf8':                           'pa_PK.UTF-8',
+    'sd_in@devanagari.utf8':                'sd_IN.UTF-8@devanagari',
+    'sr_yu.utf8':                           'sr_RS.UTF-8',
+    'sr_yu.utf8@cyrillic':                  'sr_RS.UTF-8',
+    'te_in.utf8':                           'te_IN.UTF-8',
+    'zh_sg.utf8':                           'zh_SG.UTF-8',

Some of them maps to other country (en_zw.utf8 to en_ZS.UTF-8, sr_yu.utf8 to sr_RS.UTF-8) and these mappings are different from base mappings (en_zw to en_ZW.ISO8859-1, sr_yu to sr_RS.UTF-8@latin). The devanagari mappings just maps illformed locales. c.utf8 is yet one special case. Other mappings have no base entity without encoding.

Here is a patch which enables UTF-8 mapping in makelocalealias.py and adds all these mappings to locale alias table.
msg228066 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2014-10-01 10:11
Thanks, Serhiy

The patch looks good. Please apply.
msg228120 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-10-01 21:01
After applying patch from issue20079, pa_pk.utf8, te_in.utf8, and zh_sg.utf8 are eliminated from this patch.
msg228123 - (view) Author: Roundup Robot (python-dev) Date: 2014-10-01 21:15
New changeset c4284fa75ec7 by Serhiy Storchaka in branch '2.7':
Issue #20076: Added non derived UTF-8 aliases to locale aliases table.
https://hg.python.org/cpython/rev/c4284fa75ec7

New changeset 3bff73df6dab by Serhiy Storchaka in branch '3.4':
Issue #20076: Added non derived UTF-8 aliases to locale aliases table.
https://hg.python.org/cpython/rev/3bff73df6dab

New changeset 301b9a58021c by Serhiy Storchaka in branch 'default':
Issue #20076: Added non derived UTF-8 aliases to locale aliases table.
https://hg.python.org/cpython/rev/301b9a58021c
msg228170 - (view) Author: Roundup Robot (python-dev) Date: 2014-10-02 07:23
New changeset 840af1a073f7 by Serhiy Storchaka in branch '2.7':
Issue #20076: Remove just added the sr_rs.utf8@latn alias because it is
https://hg.python.org/cpython/rev/840af1a073f7

New changeset ed92f06cdd8b by Serhiy Storchaka in branch '3.4':
Issue #20076: Remove just added the sr_rs.utf8@latn alias because it is
https://hg.python.org/cpython/rev/ed92f06cdd8b

New changeset aa169e6d2e15 by Serhiy Storchaka in branch 'default':
Issue #20076: Apply optimization in makelocalealias.py repeatedly.
https://hg.python.org/cpython/rev/aa169e6d2e15
History
Date User Action Args
2014-10-02 07:23:59python-devsetmessages: + msg228170
2014-10-01 21:30:38serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2014-10-01 21:15:05python-devsetnosy: + python-dev
messages: + msg228123
2014-10-01 21:01:02serhiy.storchakasetassignee: serhiy.storchaka
messages: + msg228120
2014-10-01 10:11:08lemburgsetmessages: + msg228066
2014-09-30 15:43:43serhiy.storchakasetversions: + Python 3.5, - Python 3.3
2013-12-27 06:11:43ezio.melottisetnosy: + ezio.melotti
2013-12-26 23:29:08serhiy.storchakacreate