classification
Title: float.__format__() handles trailing zeros inconsistently in “general” format
Type: behavior Stage: resolved
Components: Versions: Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: mark.dickinson Nosy List: davidchambers, eric.smith, mark.dickinson, miss-islington, rhettinger
Priority: normal Keywords: patch

Created on 2020-05-26 14:15 by davidchambers, last changed 2020-05-29 13:48 by mark.dickinson. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 20435 merged mark.dickinson, 2020-05-26 16:28
PR 20514 merged miss-islington, 2020-05-29 13:24
Messages (13)
msg369983 - (view) Author: David Chambers (davidchambers) Date: 2020-05-26 14:15
According to https://docs.python.org/3/library/string.html#format-specification-mini-language, “insignificant trailing zeros are removed from the significand” when 'g' is specified. I encountered a situation in which a trailing zero is not removed:

    $ python3
    Python 3.7.7 (default, Mar 10 2020, 15:43:03)
    [Clang 11.0.0 (clang-1100.0.33.17)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> '{0:.3g}'.format(1504)
    '1.5e+03'
    >>> '{0:.3g}'.format(1505)
    '1.50e+03'
    >>> '{0:.3g}'.format(1506)
    '1.51e+03'

Is this behaviour intentional? If so, why is the trailing zero in 1.50 considered significant for 1505 but insignificant for 1504?
msg369996 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-05-26 15:12
FWIW, which is probably not much with ".g" formatting, this is how Decimal behaves:

>>> from decimal import Decimal as D
>>> format(D(1504), '.3g')
'1.50e+3'
>>> format(D(1505), '.3g')
'1.50e+3'
>>> format(D(1506), '.3g')
'1.51e+3'
>>> format(D(1504.0), '.3g')
'1.50e+3'
>>> format(D(1505.0), '.3g')
'1.50e+3'
>>> format(D(1506.0), '.3g')
'1.51e+3'
>>> format(D("1504.0"), '.3g')
'1.50e+3'
>>> format(D("1505.0"), '.3g')
'1.50e+3'
>>> format(D("1506.0"), '.3g')
'1.51e+3'
msg370002 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2020-05-26 15:34
Very interesting. Agreed that this looks like a bug. It affects old-style formatting, too:

>>> "%.3g" % 1503
'1.5e+03'
>>> "%.3g" % 1504
'1.5e+03'
>>> "%.3g" % 1505
'1.50e+03'
msg370006 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2020-05-26 15:49
This appears to go all the way down to _Py_dg_dtoa, which in mode 3 is supposed to suppress trailing zeros.

I'll dig in and see if I can find a fix. (Assigning to me so that I don't forget it, but I also don't want to block anyone else - if anyone else feels like working on this, please do go ahead.)
msg370009 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-05-26 16:12
For completeness, here's the output using just format, which I prefer over str.format because there's less going on: it removes all of the str.format machinery and basically directly calls obj.__format__.

>>> format(1504, '.3g')
'1.5e+03'
>>> format(1505, '.3g')
'1.50e+03'
>>> format(1506, '.3g')
'1.51e+03'
msg370012 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2020-05-26 16:30
Created a PR with a tentative fix. It still needs regression tests; working on those.
msg370021 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2020-05-26 17:31
The PR is ready for review.
msg370025 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2020-05-26 17:37
I'm wondering how far back the fix should be backported. Clearly it should go into the 3.9 branch as well as master, but it feels like the sort of fix where the behaviour change resulting from the fix is as likely to break code as the bug itself.
msg370026 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2020-05-26 18:02
I think I'd just do 3.9 and master. It does seem subtle for a minor release, when people are less likely to be looking at the release notes.
msg370133 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-05-27 20:46
+1 for 3.9 and later.
msg370309 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2020-05-29 13:24
New changeset 895c9c1d438367722f74f437fda96767d770662b by Mark Dickinson in branch 'master':
bpo-40780: Fix failure of _Py_dg_dtoa to remove trailing zeros (GH-20435)
https://github.com/python/cpython/commit/895c9c1d438367722f74f437fda96767d770662b
msg370312 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2020-05-29 13:47
New changeset ad088ca5c6adc4a107411cad62b83f5c6e06b5ed by Miss Islington (bot) in branch '3.9':
bpo-40780: Fix failure of _Py_dg_dtoa to remove trailing zeros (GH-20435) (GH-20514)
https://github.com/python/cpython/commit/ad088ca5c6adc4a107411cad62b83f5c6e06b5ed
msg370313 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2020-05-29 13:48
Fixed in master and 3.9; not backporting to 3.8 or 3.7, as discussed.
History
Date User Action Args
2020-05-29 13:48:02mark.dickinsonsetstatus: open -> closed
resolution: fixed
messages: + msg370313

stage: patch review -> resolved
2020-05-29 13:47:02mark.dickinsonsetmessages: + msg370312
2020-05-29 13:24:16miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request19759
2020-05-29 13:24:04mark.dickinsonsetmessages: + msg370309
2020-05-27 20:46:24rhettingersetnosy: + rhettinger
messages: + msg370133
2020-05-27 13:42:42mark.dickinsonsetversions: - Python 3.7, Python 3.8
2020-05-26 20:38:43eric.smithsettitle: str.format() handles trailing zeros inconsistently in “general” format -> float.__format__() handles trailing zeros inconsistently in “general” format
2020-05-26 18:02:44eric.smithsetmessages: + msg370026
2020-05-26 17:37:33mark.dickinsonsetmessages: + msg370025
2020-05-26 17:31:25mark.dickinsonsetmessages: + msg370021
2020-05-26 16:30:09mark.dickinsonsetmessages: + msg370012
2020-05-26 16:28:34mark.dickinsonsetkeywords: + patch
stage: patch review
pull_requests: + pull_request19692
2020-05-26 16:12:53eric.smithsetmessages: + msg370009
2020-05-26 15:49:15mark.dickinsonsetassignee: mark.dickinson
messages: + msg370006
2020-05-26 15:34:30mark.dickinsonsetmessages: + msg370002
versions: + Python 3.7, Python 3.8, Python 3.9, Python 3.10
2020-05-26 15:13:16eric.smithsetnosy: + mark.dickinson
2020-05-26 15:12:32eric.smithsetmessages: + msg369996
2020-05-26 15:09:23eric.smithsetnosy: + eric.smith
2020-05-26 14:15:19davidchamberscreate