classification
Title: gettext: deprecate selecting plural form by fractional numbers
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Tim.Graham, loewis, mdk, serhiy.storchaka, xiang.zhang
Priority: normal Keywords: patch

Created on 2016-11-14 18:37 by serhiy.storchaka, last changed 2017-03-24 22:31 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
gettext-non-int-plural-deprecate.patch serhiy.storchaka, 2016-11-14 20:04 review
gettext-non-int-plural-deprecate.patch serhiy.storchaka, 2016-11-15 13:00 review
Pull Requests
URL Status Linked Edit
PR 507 merged serhiy.storchaka, 2017-03-06 10:05
Messages (11)
msg280804 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-14 18:37
GNU gettext library accepts only integer value for selecting plural form. Python gettext accepts arbitrary numbers. But gettext formulas are not purposed to support non-integer values and can return incorrect result. For example (in Ukrainian):

   "1 площа", but "1.5 площі", not "1.5 площ".
   "2 гектари", but "2.75 гектара", not "2.75 гектарів".
   "5 тонн", but "5.7 тонни", not "5.7 тонн".

Separate plural form should be used for fractional numbers. Even if fractional part happens to be zero, it is acceptable (e.g. "Time elapsed: 1.000 seconds" in English).

Proposed patch deprecates fractional numbers for selecting plural form in gettext.
msg280806 - (view) Author: Julien Palard (mdk) * Date: 2016-11-14 19:45
Hi, did you forget to attach the patch?
msg280810 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-14 20:04
Sorry. Here is a patch.
msg280821 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-11-15 02:48
Maybe you have forgotten to remove the debug print in the patch?
msg280835 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-15 13:00
Yes, of cause. Thank your for noticing this.
msg280839 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-11-15 13:53
I think the stacklevel should be 3.

stacklevel = 3:

./python -Walways /tmp/a.py 
/tmp/a.py:2: DeprecationWarning: Plural value must be an integer, got float
  c2py('n!=1')(1.1)

stacklevel = 4:

./python -Walways /tmp/a.py 
sys:1: DeprecationWarning: Plural value must be an integer, got float
msg280841 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-11-15 14:00
Ohh, sorry. It should be 4 and I make a mistake. Sorry for the noise. Patch LGTM.
msg282329 - (view) Author: Julien Palard (mdk) * Date: 2016-12-04 10:42
> But gettext formulas are not purposed to support non-integer values and can return incorrect result

Looks like the cast to integer is done *before* giving the value to the C gettext expression.

> For example (in Ukrainian)

As long as gettext expression don't support non-integer values, there exist *no* way to support two languages where plural form differ for non-integer value. Typically if a language A consider 1.5 to be singular and another language B consider 1.5 to be plural, the only place in the code that can make the difference IS in the C plural expression which don't support non-integer values.

Rouding the value before giving it to ngettext only fixes the issue for the lang of the current developer, as for other languages, translators won't be able to change the rounding properties.

But should we bet that in most, any, or all languages, 1.5 is considered plural? I think so, according to wikipedia¹: "Plural of nouns typically denote a quantity other than the default quantity represented by a noun, which is generally one."

So, I think that a better fix than warning for non-integer values may be to round them using math.ceil instead of round. This way we avoid developpers to drop round() everywhere to fix the warning the wrong way, leaving the same bug you're having in Ukrainian.

¹: https://en.wikipedia.org/wiki/Plural
msg282331 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-12-04 11:54
round() was used because it is the simplest way to convert fractional numbers to integers that doesn't involve strings-to-integer converting (as in int()). Perhaps math.trunc() looks a little more appropriate. math.ceil() returns a float in 2.7 and is limited by float range. But "1.678 second" don't look more correct than "1.678 seconds". My point is that gettext ability of selecting plural form shouldn't be used for fractional numbers. It seems to me that a fractional number should be formatted with the same form independently from it's value (it can be different from plural forms for integer numbers). Proposed patch adds a deprecating warning for encouraging users to rewrite their code for formatting fractional numbers.
msg282342 - (view) Author: Julien Palard (mdk) * Date: 2016-12-04 14:31
> It seems to me that a fractional number should be formatted with the same form independently from it's value

Ok, so 1.000 seconds, ok, LGTM.

Should we mention something explicit about it? Not sure how to write it down, but I still fear that everyone's reaction will be "Oh can't give a float? Let my put a round() in my call" and stay with the same plural bug as before.

Maybe replace

> Plural value must be an integer

by

> Should not use ngettext with floating point, consider an invariant form.

?
msg290206 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-24 22:24
New changeset f6595983e08fe20cf06a2535d74d912c6dbb044f by Serhiy Storchaka in branch 'master':
bpo-28692: Deprecate using non-integer value for selecting a plural form in gettext. (#507)
https://github.com/python/cpython/commit/f6595983e08fe20cf06a2535d74d912c6dbb044f
History
Date User Action Args
2017-03-24 22:31:54serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-03-24 22:24:01serhiy.storchakasetmessages: + msg290206
2017-03-18 06:32:17serhiy.storchakasetpull_requests: - pull_request605
2017-03-17 21:00:34larrysetpull_requests: + pull_request605
2017-03-06 10:05:05serhiy.storchakasetpull_requests: + pull_request416
2016-12-04 14:31:10mdksetmessages: + msg282342
2016-12-04 11:54:52serhiy.storchakasetmessages: + msg282331
2016-12-04 10:42:25mdksetmessages: + msg282329
2016-11-15 14:00:45xiang.zhangsetmessages: + msg280841
2016-11-15 13:53:31xiang.zhangsetmessages: + msg280839
2016-11-15 13:00:28serhiy.storchakasetfiles: + gettext-non-int-plural-deprecate.patch

messages: + msg280835
2016-11-15 02:48:18xiang.zhangsetnosy: + xiang.zhang
messages: + msg280821
2016-11-14 20:04:24serhiy.storchakasetfiles: + gettext-non-int-plural-deprecate.patch
keywords: + patch
messages: + msg280810
2016-11-14 19:45:02mdksetnosy: + mdk
messages: + msg280806
2016-11-14 18:37:59serhiy.storchakacreate