classification
Title: 2.7.10 test__locale.py change breaks on Solaris
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6, Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: jbeck, lemburg, loewis, python-dev, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2015-05-27 15:43 by jbeck, last changed 2015-06-08 18:31 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
25-test__locale.patch jbeck, 2015-05-27 15:43 patch for test__locale.py to fix fr_FR thousand_sep
issue24299-2.7.patch serhiy.storchaka, 2015-05-27 18:25 review
Messages (5)
msg244181 - (view) Author: John Beck (jbeck) Date: 2015-05-27 15:43
The upgrade from 2.7.9 to 2.7.10 resulted in test__locale failing.
This test had previously succeeded.  The difference is that the
thousands-separator for the fr_FR locale in known_numerics was
changed from '' (i.e., unknown) to ' ' (i.e. space).  But on Solaris,
'\xa0' (i.e., non-break space in ISO8859-1) is what the fr_FR locale
returns for LC_NUMERIC's thousands-separator.  I inquired with our
Globalization experts, who replied:

---
The short answer is that CLDR defines the group separator as no-break 
space (U+00A0): http://st.unicode.org/cldr-apps/v#/fr/Symbols/
so the solaris locale fr_FR (=fr_FR.ISO8859-1) is correct.

The long answer is that the situation is confusing, the fr_FR.ISO8859-1
defines the thousands_sep as no-break space, but fr_FR.UTF-8 defines
the thousands_sep as space (U+0020). There is no technical limit, but
combination of POSIX [1] and C language [2] limits the thousands_sep
to single byte character. The no-break space is single byte character
in ISO8859-1, but multibyte in UTF-8.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap07.html#tag_07_03_04

[2] http://en.cppreference.com/w/c/locale/lconv
    &&
    http://en.cppreference.com/w/c/language/character_constant
---
The attached patch fixes the test on Solaris.  It is not clear if this
is the Right Answer for all platforms, but I offer the attached patch
in case it helps anyone else.
msg244212 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-05-27 18:25
This is not the Right Answer because on Linux the thousands-separator for the fr_FR locale is a space.

Perhaps better solution would be to specify the UTF-8 encoding for fr_FR locale.
msg244993 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-06-08 08:36
Does the patch fix the issue John?
msg245002 - (view) Author: John Beck (jbeck) Date: 2015-06-08 12:25
(Apologies for not responding on May 27 when you posted the patch;
I failed to notice the "Added file:" line in the e-mail notification.)

Yes!  The patch you posted fixes the issue.  Thank you!
msg245010 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-06-08 15:50
New changeset 00f8804f559c by Serhiy Storchaka in branch '2.7':
Issue #24299: Fixed test__locale on Solaris.
https://hg.python.org/cpython/rev/00f8804f559c

New changeset da642b5aaf79 by Serhiy Storchaka in branch '3.4':
Issue #24299: Fixed test__locale on Solaris.
https://hg.python.org/cpython/rev/da642b5aaf79

New changeset 0a1f1988fece by Serhiy Storchaka in branch '3.5':
Issue #24299: Fixed test__locale on Solaris.
https://hg.python.org/cpython/rev/0a1f1988fece

New changeset 30d8b5b62245 by Serhiy Storchaka in branch 'default':
Issue #24299: Fixed test__locale on Solaris.
https://hg.python.org/cpython/rev/30d8b5b62245
History
Date User Action Args
2015-06-08 18:31:10serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2015-06-08 15:50:54python-devsetnosy: + python-dev
messages: + msg245010
2015-06-08 12:25:56jbecksetmessages: + msg245002
2015-06-08 08:36:49serhiy.storchakasetmessages: + msg244993
2015-05-27 18:25:24serhiy.storchakasetfiles: + issue24299-2.7.patch

assignee: serhiy.storchaka
versions: + Python 3.4, Python 3.5, Python 3.6
nosy: + lemburg, loewis, serhiy.storchaka

messages: + msg244212
stage: patch review
2015-05-27 15:43:31jbeckcreate