classification
Title: locale.atof documentation is missing func argument
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: ced, docs@python, ezio.melotti, georg.brandl, mark.dickinson, pitrou, python-dev, terry.reedy, vstinner
Priority: normal Keywords: patch

Created on 2012-02-01 14:12 by ced, last changed 2014-10-23 21:04 by georg.brandl. This issue is now closed.

Files
File name Uploaded Description Edit
doc_atof.patch ced, 2013-03-15 09:46 review
delocalize.patch ced, 2014-10-17 09:32 review
delocalize.patch ced, 2014-10-18 13:13 review
delocalize.patch ced, 2014-10-18 21:08 review
Messages (24)
msg152430 - (view) Author: Cédric Krier (ced) * Date: 2012-02-01 14:12
atof has a func argument used to instantiate the result but it is missing in the documentation.
msg152478 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-02-02 21:29
I don't think that argument needs to be documented.  It's just there because somebody thought that copying 3 lines from atof into atoi was a bad idea.
msg152479 - (view) Author: Cédric Krier (ced) * Date: 2012-02-02 21:36
Indeed I find it useful to use to get a Decimal instead of a float.
So I was wondering if I can rely on it or not in my application?
msg184119 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-03-13 23:23
It looks like an implementation detail to me, so I tend to agree with Georg.  I'm not sure if this should be noted in the code though.
msg184121 - (view) Author: Cédric Krier (ced) * Date: 2013-03-14 00:03
Then I think we miss a locale.atod to parse string to Decimal
msg184165 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-03-14 14:37
I agree with "won't fix" for the original issue. These locale functions
are in effect superseded by PEP 3101 formatting.

For decimal locale specific formatting, use:

format(Decimal("1729.1415927"), "n")


IOW, I don't think new formatting functions should be added to the
locale module.
msg184167 - (view) Author: Cédric Krier (ced) * Date: 2013-03-14 15:05
locale.atof is not about formatting but parsing string into float following the locale.
For now, the only ways I see to parse a string to get a Decimal is to first convert it into float (which is not good if precision matters) or to use the undocumented parameter.
msg184168 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-03-14 15:44
Cédric Krier <report@bugs.python.org> wrote:
> locale.atof is not about formatting but parsing string into float following the locale.

You're right. Sorry, I never use these locale functions. My impression is
that locales are often buggy or differ across platforms (see #16944).

So actually I now agree that making the parameter official is one reasonable
solution.
msg184226 - (view) Author: Cédric Krier (ced) * Date: 2013-03-15 09:46
Here is a patch for the documentation.
msg184251 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-03-15 20:21
I find the idea of intentionally not documenting a public parameter and the full signature of a function somewhat strange, especially when it is already automatically partially-documented.

>>> import locale
>>> help(locale.atof)
Help on function atof in module locale:

atof(string, func=<class 'float'>)
    Parses a string as a float according to the locale settings.
# 2.7, 3.2, 3.3

Not documenting the full signature of a function seems to me contrary to proper policy. That aside, the func parameter is, to me, a useful feature, not just an implementation detail

The way to have factored out the common normalization without a func parameter is obvious: define a private normalization function.

def _anormalize(string):
    "remove thousands separators, make decimal dot"
    ts = localeconv()['thousands_sep']
    if ts:
        string = string.replace(ts, '')
    #next, replace the decimal point with a dot
    dd = localeconv()['decimal_point']
    if dd:
        string = string.replace(dd, '.')
    return string

def atof(string):
    "Parses a string as a float according to the locale settings."
    return float(_anormalize(string))

def atoi(string):  # changed from str
    "Converts a string to an integer according to the locale settings."
    return int(_anormalize(string))

But Martin von Loewis, the original author did not do this. I would not assume that he "thought that copying 3 lines from atof into atoi was a bad idea." without asking him.  Whatever his conscious intention, the func parameter *does* have the advantage of allowing alternate float string to number converters.  We now have another one in the stdlib besides decimal.Decimal: fractions.Fractions.

>>> locale.atof('99,999.99', F)
Fraction(9999999, 100)
# versus
>>> F(locale.atof('99,999.99'))
Fraction(6871946986405233, 68719476736)

There are also 3rd party float implementations, such as indefinite precision binary floats. Does anyone still object to properly documenting this useful feature? I am willing to do the commits.

As to the patch and atof docstring, I thinks 'converts' (used in atoi docstring) is better than 'parses'.  So I would change both.
msg184292 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-03-16 05:16
The function was introduced by Guido in f5b55311e79d.

I think it would have been better if atof had another name (e.g. _atof) and that atof and atoi were implemented as:

def atof(str):
    return _atof(str, float)
def atoi(str):
    return _atof(str, int)

Even better would have been to have _atof return the string without applying any function, and let e.g. atoi return int(_atof(str)).  This could have been documented publicly and used to build other functions.
However it's probably too late for such refactoring.
msg184295 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-03-16 06:29
The refactoring could be done if we were willing to give the normalize function a public name, so people could write Decimal(delocalize(<localized float string>)) or if we were willing to add atod and atofr (fraction). However, simply adding a few words to the doc is a lot easier.
msg184612 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-03-19 07:17
It's easier, but we will be exposing an API that is not too elegant IMHO.  The refactoring could provide a better API, but OTOH it will make it available for 3.4+ only, and it would break backward compatibility if the old API is removed (even though it's not documented, so in theory we could do that).
msg229570 - (view) Author: Cédric Krier (ced) * Date: 2014-10-17 09:32
So what about this patch?
It adds a delocalize method while keeping the atof func parameter for backward compatibility.
msg229573 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-17 13:38
The patch's approach looks reasonable to me.
msg229641 - (view) Author: Cédric Krier (ced) * Date: 2014-10-18 13:13
A new version with unittest.
msg229644 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-18 16:50
Thanks for the updated patch, Cédric. Just some remarks:

- the documentation should mention that the return value is still a string
- the documentation needs a "versionadded" marker next to the new function
- have you already signed the contributor's agreement? otherwise you should sign it at https://www.python.org/psf/contrib/contrib-form/
msg229660 - (view) Author: Cédric Krier (ced) * Date: 2014-10-18 21:08
Add return value is string in doc
Add versionadded
And yes I signed the agreement.
msg229899 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-23 20:49
Thank you! The patch looks good to me, I'm going to apply it.
msg229901 - (view) Author: Roundup Robot (python-dev) Date: 2014-10-23 20:52
New changeset aee097e5a2b2 by Antoine Pitrou in branch 'default':
Issue #13918: Provide a locale.delocalize() function which can remove
https://hg.python.org/cpython/rev/aee097e5a2b2
msg229902 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-23 20:53
Done. Thank you for your contribution!
msg229903 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-10-23 20:58
+    :const:'LC_NUMERIC`settings.

a space is missing before "settings", no?
msg229905 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-23 21:03
Ah, right, thank you.
msg229906 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-10-23 21:04
And the first quote is wrong.
History
Date User Action Args
2014-10-23 21:04:24georg.brandlsetmessages: + msg229906
2014-10-23 21:03:58pitrousetmessages: + msg229905
2014-10-23 20:58:17vstinnersetnosy: + vstinner
messages: + msg229903
2014-10-23 20:53:30pitrousetstatus: open -> closed
resolution: fixed
messages: + msg229902

stage: patch review -> resolved
2014-10-23 20:52:52python-devsetnosy: + python-dev
messages: + msg229901
2014-10-23 20:49:42pitrousetmessages: + msg229899
2014-10-18 21:08:33cedsetfiles: + delocalize.patch

messages: + msg229660
2014-10-18 16:50:34pitrousetmessages: + msg229644
2014-10-18 13:13:19cedsetfiles: + delocalize.patch

messages: + msg229641
2014-10-17 13:38:34pitrousetnosy: + pitrou
messages: + msg229573
2014-10-17 13:37:52pitrousetassignee: docs@python ->
stage: patch review
components: + Library (Lib), - Documentation
versions: + Python 3.5, - Python 2.7, Python 3.2, Python 3.3, Python 3.4
2014-10-17 09:32:12cedsetfiles: + delocalize.patch

messages: + msg229570
2014-10-14 15:12:57skrahsetnosy: - skrah
2013-03-19 07:17:18ezio.melottisetmessages: + msg184612
2013-03-16 06:29:08terry.reedysetmessages: + msg184295
2013-03-16 05:16:18ezio.melottisetmessages: + msg184292
2013-03-15 20:21:07terry.reedysetnosy: + terry.reedy

messages: + msg184251
versions: + Python 2.7, Python 3.2, Python 3.3, Python 3.4
2013-03-15 09:46:35cedsetfiles: + doc_atof.patch
resolution: wont fix -> (no value)
messages: + msg184226

keywords: + patch
2013-03-14 15:44:12skrahsetmessages: + msg184168
2013-03-14 15:05:41cedsetstatus: closed -> open

messages: + msg184167
2013-03-14 14:37:43skrahsetstatus: open -> closed

messages: + msg184165
2013-03-14 00:33:34ezio.melottisetnosy: + mark.dickinson, skrah
2013-03-14 00:03:19cedsetstatus: pending -> open

messages: + msg184121
2013-03-13 23:23:26ezio.melottisetstatus: open -> pending
nosy: + ezio.melotti
messages: + msg184119

2012-02-02 21:36:30cedsetstatus: pending -> open

messages: + msg152479
2012-02-02 21:29:14georg.brandlsetstatus: open -> pending

nosy: + georg.brandl
messages: + msg152478

resolution: wont fix
2012-02-01 14:12:20cedcreate