This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_locale fails with Turkish locale
Type: behavior Stage: resolved
Components: Library (Lib), Tests Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: David.Edelsohn, berker.peksag, lemburg, loewis, python-dev, serhiy.storchaka, skrah
Priority: normal Keywords: patch

Created on 2013-12-28 09:00 by serhiy.storchaka, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
locale_tr_TR.patch serhiy.storchaka, 2014-01-10 23:03 review
Messages (10)
msg207024 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-12-28 09:00
See http://buildbot.python.org/all/builders/PPC64%20PowerLinux%202.7/builds/396/steps/test/logs/stdio.

======================================================================
ERROR: test_getsetlocale_issue1813 (test.test_locale.TestMiscellaneous)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/2.7.edelsohn-powerlinux-ppc64/build/Lib/test/test_locale.py", line 480, in test_getsetlocale_issue1813
    locale.setlocale(locale.LC_CTYPE, loc)
  File "/home/shager/cpython-buildarea/2.7.edelsohn-powerlinux-ppc64/build/Lib/locale.py", line 579, in setlocale
    return _setlocale(category, locale)
Error: unsupported locale setting

----------------------------------------------------------------------
msg207027 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-12-28 12:32
I don't know why this fails exactly. I had a similar failure though that was fixed by using the @run_with_locale decorator.
msg207859 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-01-10 13:35
New changeset 3696b9ae6b17 by Serhiy Storchaka in branch '2.7':
Issue #20086: Output more details when test_getsetlocale_issue1813 is failed.
http://hg.python.org/cpython/rev/3696b9ae6b17
msg207870 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-10 17:09
Detailed output:
http://buildbot.python.org/all/builders/PPC64%20PowerLinux%202.7/builds/415/steps/test/logs/stdio
======================================================================
FAIL: test_getsetlocale_issue1813 (test.test_locale.TestMiscellaneous)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/2.7.edelsohn-powerlinux-ppc64/build/Lib/test/test_locale.py", line 484, in test_getsetlocale_issue1813
    (loc, oldlocale, e))
AssertionError: Failed to set locale ('tr_TR', 'ISO8859-9') (default locale is (None, None)): Error('unsupported locale setting',)

----------------------------------------------------------------------

locale.getlocale() calls _parse_localename() which uses normalize() which maps 'tr_TR' to 'tr_TR.ISO8859-9' with according to locale_alias table. However on this platform tr_TR.ISO8859-9 doesn't not exist, only tr_TR.
msg207882 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-10 21:01
This bug is occured not only on PPC64 PowerLinux. See also:

http://buildbot.python.org/all/builders/x86%20RHEL%206%202.7/builds/975/steps/test/logs/stdio
http://buildbot.python.org/all/builders/AMD64%20Fedora%20without%20threads%202.7/builds/1644/steps/test/logs/stdio
msg207884 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-01-10 22:29
3d805bee06e2 uses str.lower(), which fails on the Turkish 'i' if the Turkish locale is set:


Python 2.7.6+ (2.7:0e5df5b62488+, Jan 10 2014, 23:25:35) 
[GCC 4.6.3 20120306 (Red Hat 4.6.3-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.setlocale(locale.LC_CTYPE, ('tr_TR', 'ISO8859-9'))
'tr_TR.ISO8859-9'
>>> 'tr_TR.ISO8859-9'.lower()
'tr_tr.\xfdso8859-9'
msg207885 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-01-10 22:43
I'm not sure if str.lower() is supposed to use tolower() in
stringobject.c. If tolower() is replaced by Py_TOLOWER(), the
problem disappears.

But maybe the locale dependent tolower() is used on purpose.
msg207886 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-01-10 22:49
Ignore me, the fine manual says that "For 8-bit strings, this method is
locale-dependent."
msg207889 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-10 23:03
Good catch, Stefan!

Here is a patch which restores old behavior. It also changes test_locale to try several Turkish locale names. On some platforms (as Ubuntu) bare tr_TR is not exist, but only tr_TK.utf8. This will increase chance for this test to run.
msg208323 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-01-17 07:28
New changeset bac50f54d357 by Serhiy Storchaka in branch '2.7':
Issue #20086: Restored the use of locale-independing mapping instead of
http://hg.python.org/cpython/rev/bac50f54d357
History
Date User Action Args
2022-04-11 14:57:56adminsetgithub: 64285
2014-01-17 08:18:30serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2014-01-17 07:28:57python-devsetmessages: + msg208323
2014-01-10 23:03:47serhiy.storchakasetfiles: + locale_tr_TR.patch
messages: + msg207889

assignee: serhiy.storchaka
keywords: + patch
stage: patch review
2014-01-10 22:49:08skrahsetmessages: + msg207886
2014-01-10 22:43:50skrahsetmessages: + msg207885
2014-01-10 22:29:14skrahsetmessages: + msg207884
2014-01-10 21:19:58berker.peksagsetnosy: + berker.peksag
2014-01-10 21:02:47serhiy.storchakasettitle: test_locale fails on PPC64 PowerLinux -> test_locale fails with Turkish locale
2014-01-10 21:01:25serhiy.storchakasetmessages: + msg207882
2014-01-10 17:09:33serhiy.storchakasetmessages: + msg207870
2014-01-10 13:35:34python-devsetnosy: + python-dev
messages: + msg207859
2013-12-28 12:32:17skrahsetnosy: + skrah
messages: + msg207027
2013-12-28 10:28:37pitrousetnosy: + David.Edelsohn
2013-12-28 09:00:19serhiy.storchakacreate