This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: "Invalid" tests on locales
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.6, Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: bapt, koobs, larry, lemburg, loewis, martin.panter, python-dev, serhiy.storchaka, vstinner, zach.ware
Priority: normal Keywords: buildbot, patch

Created on 2015-12-05 17:59 by bapt, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
skip-thousands.patch martin.panter, 2015-12-11 04:04 review
Messages (21)
msg255964 - (view) Author: bapt (bapt) Date: 2015-12-05 17:59
the locale tests the return value of locale functions against known good values.

The problem is when those known good values becomes wrong because of an update of the locales.

For example in recent CLDR definition the french separator for thousands in a "non breaking space" and not a space, while it used to be a space.
On released FreeBSD (very old db) it was a space. On GNU libc (a more recent db but still old :)) it is a space.On FreeBSD 11 and Dragonfly 4.4 it is a fairly new db (v27.0.1) with the fixed separator: 0xa0!

Thant makes this test fail.

I have no idea on the way you would prefer to fix that, either remove the test (not sure it is really relevant in the python testsuite) or patch it.

https://hg.python.org/cpython/file/tip/Lib/test/test__locale.py#l70
the failures can be seen here:
http://buildbot.python.org/all/buildslaves/koobs-freebsd11
msg255965 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2015-12-05 18:13
I see three options:

1. remove the French entry from the dict

2. enhance the test to accept multiple valid values

3. replace the French entry with a different locale that doesn't change the mapping often
msg255978 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-12-05 21:12
Originally the case for French was 'fr_FR': (',', ''). '' means "unknown" value and is not tested. We can return this value to omit test for thousand separator, or remove the French locale at all.
msg256037 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-12-07 00:00
Some history:

1. Issue 23474 (Feb 2015): Testing fr_FR thousands_sep = ASCII space

2. Issue 24299 (Jun 2015): Changed to fr_FR.UTF-8 because Solaris uses the non-breaking space if it can be encoded as a single byte (i.e Latin-1)

So I tend to agree with Serhiy about changing back to the untested empty string. I don’t see the need to completely remove the whole test though.
msg256188 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-12-11 04:04
Here is a patch that reverts to plain fr_FR and skips testing the thousands separator. This is what was tested before revision f9ff2a5bbbe2 (Issue 23474).
msg256191 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-12-11 04:25
I meant revision b53aadd9cf85 in above message
msg256202 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-12-11 06:39
LGTM.
msg256208 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2015-12-11 08:36
Same here. Thanks Martin.
msg256270 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-12-12 07:31
New changeset 4f24a6dc934b by Martin Panter in branch '3.5':
Issue #25809: Skip testing platform-dependent French thousands separator
https://hg.python.org/cpython/rev/4f24a6dc934b

New changeset 7c5c03143923 by Martin Panter in branch 'default':
Issue #25809: Merge French locale test from 3.5
https://hg.python.org/cpython/rev/7c5c03143923
msg256272 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-12-12 07:41
New changeset 903a2664d32d by Martin Panter in branch '2.7':
Issue #25809: Skip testing platform-dependent French thousands separator
https://hg.python.org/cpython/rev/903a2664d32d
msg256273 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-12-12 08:22
Thank you Martin.
msg256279 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-12-12 11:28
The koobs-freebsd11 buildbots are starting to look greener now
msg256288 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-12-12 17:05
I would commit the patch on 3.4 too. Patches that just fixes tests or documentation can be applied even at this stage.
msg256297 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-12-12 22:30
Okay I am happy to do that, but I am unsure of the procedure. Do I just commit to 3.4, merge forward, and push? Would this be picked up in the final 3.4.4 release, or only in a potential 3.4.5 release? Would I have to check with Larry first?
msg256584 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2015-12-17 06:24
Re-open for 3.4 merge
msg256585 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-12-17 06:43
Just commit to 3.4, merge forward, and push. I don't think there is need to pick up this in the final 3.4.4 release. But this can be picked up by maintainers of specific Linux distributions if new libc will come with the non-breaking space for French locale. Looks as there is a tendency to move from the space to the non-breaking space.
msg256586 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2015-12-17 07:07
Just to note, the koobs-freebsd-current buildbot is still failing on 3.4, which reminded me to re-open
msg256596 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-12-17 10:37
New changeset bb7a19e42566 by Martin Panter in branch '3.4':
Issue #25809: Skip testing platform-dependent French thousands separator
https://hg.python.org/cpython/rev/bb7a19e42566
msg256600 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-12-17 12:08
3.4 buildbot is fixed now
msg256602 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-12-17 12:15
Thanks Martin.
msg256603 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2015-12-17 12:39
Thank you all. A+++, would collaborate again.
History
Date User Action Args
2022-04-11 14:58:24adminsetgithub: 69995
2015-12-17 12:39:57koobssetmessages: + msg256603
2015-12-17 12:15:39serhiy.storchakasetmessages: + msg256602
2015-12-17 12:08:15martin.pantersetstatus: open -> closed

messages: + msg256600
2015-12-17 10:37:10python-devsetmessages: + msg256596
2015-12-17 07:08:00koobssetmessages: + msg256586
2015-12-17 06:43:14serhiy.storchakasetnosy: + larry
messages: + msg256585
2015-12-17 06:24:02koobssetstatus: closed -> open

messages: + msg256584
versions: + Python 3.4
2015-12-12 22:30:29martin.pantersetmessages: + msg256297
2015-12-12 17:05:31serhiy.storchakasetmessages: + msg256288
2015-12-12 11:28:47martin.pantersetstatus: open -> closed
resolution: fixed
messages: + msg256279

stage: commit review -> resolved
2015-12-12 08:22:58serhiy.storchakasetmessages: + msg256273
2015-12-12 07:41:38python-devsetmessages: + msg256272
2015-12-12 07:31:27python-devsetnosy: + python-dev
messages: + msg256270
2015-12-11 08:36:20lemburgsetmessages: + msg256208
2015-12-11 06:39:44serhiy.storchakasetmessages: + msg256202
stage: patch review -> commit review
2015-12-11 04:25:55martin.pantersetmessages: + msg256191
2015-12-11 04:04:18martin.pantersetfiles: + skip-thousands.patch
keywords: + patch
messages: + msg256188

stage: needs patch -> patch review
2015-12-07 00:00:31martin.pantersetnosy: + martin.panter
messages: + msg256037
2015-12-06 00:38:46koobssetnosy: + koobs
2015-12-05 21:12:10serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg255978
2015-12-05 18:34:04zach.waresetkeywords: + buildbot
nosy: + zach.ware
stage: needs patch

versions: - Python 3.2, Python 3.3, Python 3.4
2015-12-05 18:13:34lemburgsetmessages: + msg255965
2015-12-05 18:06:32SilentGhostsetnosy: + lemburg, loewis
2015-12-05 17:59:21baptcreate