classification
Title: wcscoll is broken on Android and test_locale fails
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: xdegaye Nosy List: Chi Hsuan Yen, python-dev, xdegaye
Priority: normal Keywords: patch

Created on 2016-12-17 08:35 by xdegaye, last changed 2016-12-19 10:16 by xdegaye. This issue is now closed.

Files
File name Uploaded Description Edit
test_locale_strcoll.patch xdegaye, 2016-12-17 16:02 review
Messages (5)
msg283474 - (view) Author: Xavier de Gaye (xdegaye) * (Python committer) Date: 2016-12-17 08:35
These failures happen now that issue 28596 has been fixed and that locale.getpreferredencoding(False) returns 'UTF-8'.

======================================================================
FAIL: test_strcoll_with_diacritic (test.test_locale.TestEnUSCollation)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_locale.py", line 362, in test_strcoll_with_diacritic
    self.assertLess(locale.strcoll('à', 'b'), 0)
AssertionError: 1 not less than 0

======================================================================
FAIL: test_strxfrm_with_diacritic (test.test_locale.TestEnUSCollation)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_locale.py", line 365, in test_strxfrm_with_diacritic
    self.assertLess(locale.strxfrm('à'), locale.strxfrm('b'))
AssertionError: 'à' not less than 'b'

----------------------------------------------------------------------
msg283496 - (view) Author: Xavier de Gaye (xdegaye) * (Python committer) Date: 2016-12-17 15:53
Both strcoll() and strxfrm() are broken (character 'à' unicode code point is 'e0'):

>>> import locale
>>> locale.setlocale(locale.LC_ALL, 'en_US.UTF-8')
'C.UTF-8'
>>> locale.strcoll('\u00e0', 'b')
1
>>> locale.strxfrm('\u00e0') < locale.strxfrm('b')
False

The correct results are -1 and True.
msg283497 - (view) Author: Xavier de Gaye (xdegaye) * (Python committer) Date: 2016-12-17 16:02
Here is a patch that skips test_strcoll_with_diacritic and test_strxfrm_with_diacritic.
msg283500 - (view) Author: Chi Hsuan Yen (Chi Hsuan Yen) * Date: 2016-12-17 17:09
I'm afraid that the sentence "wcscoll/wcsxfrm have known bugs" is misleading for people who are not quite familiar with Android. The actual cause is that BioniC's setlocale() behaves differently than other platforms. Most implementations returns NULL if en_US.UTF-8 is not available, but BioniC returns C.UTF-8. I guess it's better to add some comments for the real cause on Android.
msg283598 - (view) Author: Roundup Robot (python-dev) Date: 2016-12-19 09:47
New changeset 781c56168484 by Xavier de Gaye in branch '3.6':
Issue #28996: Skip two tests that fail on Android with the locale strcoll() and
https://hg.python.org/cpython/rev/781c56168484

New changeset 5c5cf7687dc1 by Xavier de Gaye in branch 'default':
Issue #28996: Merge 3.6.
https://hg.python.org/cpython/rev/5c5cf7687dc1
History
Date User Action Args
2016-12-19 10:16:19xdegayesetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2016-12-19 09:47:34python-devsetnosy: + python-dev
messages: + msg283598
2016-12-17 17:09:30Chi Hsuan Yensetnosy: + Chi Hsuan Yen
messages: + msg283500
2016-12-17 16:02:25xdegayesetfiles: + test_locale_strcoll.patch
keywords: + patch
messages: + msg283497

stage: needs patch -> patch review
2016-12-17 15:53:31xdegayesetmessages: + msg283496
versions: + Python 3.6
2016-12-17 09:06:06xdegayelinkissue26865 dependencies
2016-12-17 08:35:18xdegayecreate