Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

float.__format__() handles trailing zeros inconsistently in “general” format #84957

Closed
davidchambers mannequin opened this issue May 26, 2020 · 13 comments
Closed

float.__format__() handles trailing zeros inconsistently in “general” format #84957

davidchambers mannequin opened this issue May 26, 2020 · 13 comments
Assignees
Labels
3.9 only security fixes 3.10 only security fixes type-bug An unexpected behavior, bug, or error

Comments

@davidchambers
Copy link
Mannequin

davidchambers mannequin commented May 26, 2020

BPO 40780
Nosy @rhettinger, @mdickinson, @ericvsmith, @davidchambers, @miss-islington
PRs
  • bpo-40780: Fix failure of _Py_dg_dtoa to remove trailing zeros #20435
  • [3.9] bpo-40780: Fix failure of _Py_dg_dtoa to remove trailing zeros (GH-20435) #20514
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/mdickinson'
    closed_at = <Date 2020-05-29.13:48:02.325>
    created_at = <Date 2020-05-26.14:15:19.529>
    labels = ['type-bug', '3.9', '3.10']
    title = 'float.__format__() handles trailing zeros inconsistently in \xe2\x80\x9cgeneral\xe2\x80\x9d format'
    updated_at = <Date 2020-05-29.13:48:02.324>
    user = 'https://github.com/davidchambers'

    bugs.python.org fields:

    activity = <Date 2020-05-29.13:48:02.324>
    actor = 'mark.dickinson'
    assignee = 'mark.dickinson'
    closed = True
    closed_date = <Date 2020-05-29.13:48:02.325>
    closer = 'mark.dickinson'
    components = []
    creation = <Date 2020-05-26.14:15:19.529>
    creator = 'davidchambers'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 40780
    keywords = ['patch']
    message_count = 13.0
    messages = ['369983', '369996', '370002', '370006', '370009', '370012', '370021', '370025', '370026', '370133', '370309', '370312', '370313']
    nosy_count = 5.0
    nosy_names = ['rhettinger', 'mark.dickinson', 'eric.smith', 'davidchambers', 'miss-islington']
    pr_nums = ['20435', '20514']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue40780'
    versions = ['Python 3.9', 'Python 3.10']

    @davidchambers
    Copy link
    Mannequin Author

    davidchambers mannequin commented May 26, 2020

    According to https://docs.python.org/3/library/string.html#format-specification-mini-language, “insignificant trailing zeros are removed from the significand” when 'g' is specified. I encountered a situation in which a trailing zero is not removed:

        $ python3
        Python 3.7.7 (default, Mar 10 2020, 15:43:03)
        [Clang 11.0.0 (clang-1100.0.33.17)] on darwin
        Type "help", "copyright", "credits" or "license" for more information.
        >>> '{0:.3g}'.format(1504)
        '1.5e+03'
        >>> '{0:.3g}'.format(1505)
        '1.50e+03'
        >>> '{0:.3g}'.format(1506)
        '1.51e+03'

    Is this behaviour intentional? If so, why is the trailing zero in 1.50 considered significant for 1505 but insignificant for 1504?

    @davidchambers davidchambers mannequin added type-bug An unexpected behavior, bug, or error labels May 26, 2020
    @ericvsmith
    Copy link
    Member

    FWIW, which is probably not much with ".g" formatting, this is how Decimal behaves:

    >>> from decimal import Decimal as D
    >>> format(D(1504), '.3g')
    '1.50e+3'
    >>> format(D(1505), '.3g')
    '1.50e+3'
    >>> format(D(1506), '.3g')
    '1.51e+3'
    >>> format(D(1504.0), '.3g')
    '1.50e+3'
    >>> format(D(1505.0), '.3g')
    '1.50e+3'
    >>> format(D(1506.0), '.3g')
    '1.51e+3'
    >>> format(D("1504.0"), '.3g')
    '1.50e+3'
    >>> format(D("1505.0"), '.3g')
    '1.50e+3'
    >>> format(D("1506.0"), '.3g')
    '1.51e+3'

    @mdickinson
    Copy link
    Member

    Very interesting. Agreed that this looks like a bug. It affects old-style formatting, too:

    >>> "%.3g" % 1503
    '1.5e+03'
    >>> "%.3g" % 1504
    '1.5e+03'
    >>> "%.3g" % 1505
    '1.50e+03'

    @mdickinson mdickinson added 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes labels May 26, 2020
    @mdickinson
    Copy link
    Member

    This appears to go all the way down to _Py_dg_dtoa, which in mode 3 is supposed to suppress trailing zeros.

    I'll dig in and see if I can find a fix. (Assigning to me so that I don't forget it, but I also don't want to block anyone else - if anyone else feels like working on this, please do go ahead.)

    @ericvsmith
    Copy link
    Member

    For completeness, here's the output using just format, which I prefer over str.format because there's less going on: it removes all of the str.format machinery and basically directly calls obj.__format__.

    >>> format(1504, '.3g')
    '1.5e+03'
    >>> format(1505, '.3g')
    '1.50e+03'
    >>> format(1506, '.3g')
    '1.51e+03'

    @mdickinson
    Copy link
    Member

    Created a PR with a tentative fix. It still needs regression tests; working on those.

    @mdickinson
    Copy link
    Member

    The PR is ready for review.

    @mdickinson
    Copy link
    Member

    I'm wondering how far back the fix should be backported. Clearly it should go into the 3.9 branch as well as master, but it feels like the sort of fix where the behaviour change resulting from the fix is as likely to break code as the bug itself.

    @ericvsmith
    Copy link
    Member

    I think I'd just do 3.9 and master. It does seem subtle for a minor release, when people are less likely to be looking at the release notes.

    @ericvsmith ericvsmith changed the title str.format() handles trailing zeros inconsistently in “general” format float.__format__() handles trailing zeros inconsistently in “general” format May 26, 2020
    @ericvsmith ericvsmith changed the title str.format() handles trailing zeros inconsistently in “general” format float.__format__() handles trailing zeros inconsistently in “general” format May 26, 2020
    @mdickinson mdickinson removed 3.7 (EOL) end of life 3.8 only security fixes labels May 27, 2020
    @rhettinger
    Copy link
    Contributor

    +1 for 3.9 and later.

    @mdickinson
    Copy link
    Member

    New changeset 895c9c1 by Mark Dickinson in branch 'master':
    bpo-40780: Fix failure of _Py_dg_dtoa to remove trailing zeros (GH-20435)
    895c9c1

    @mdickinson
    Copy link
    Member

    New changeset ad088ca by Miss Islington (bot) in branch '3.9':
    bpo-40780: Fix failure of _Py_dg_dtoa to remove trailing zeros (GH-20435) (GH-20514)
    ad088ca

    @mdickinson
    Copy link
    Member

    Fixed in master and 3.9; not backporting to 3.8 or 3.7, as discussed.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes 3.10 only security fixes type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants