Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coverity scan: Python/dtoa.c resource leak #80443

Closed
stratakis mannequin opened this issue Mar 11, 2019 · 8 comments
Closed

Coverity scan: Python/dtoa.c resource leak #80443

stratakis mannequin opened this issue Mar 11, 2019 · 8 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@stratakis
Copy link
Mannequin

stratakis mannequin commented Mar 11, 2019

BPO 36262
Nosy @mdickinson, @vstinner, @stratakis
PRs
  • bpo-36262: Fix _Py_dg_strtod() memory leak (goto undfl) #12276
  • [3.7] bpo-36262: Fix _Py_dg_strtod() memory leak (goto undfl) (GH-12276) #12331
  • [2.7] bpo-36262: Fix _Py_dg_strtod() memory leak (goto undfl) (GH-12276) #12332
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-03-14.16:20:37.494>
    created_at = <Date 2019-03-11.13:53:27.342>
    labels = ['interpreter-core', '3.7', '3.8']
    title = 'Coverity scan: Python/dtoa.c resource leak'
    updated_at = <Date 2019-03-14.16:20:37.493>
    user = 'https://github.com/stratakis'

    bugs.python.org fields:

    activity = <Date 2019-03-14.16:20:37.493>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-03-14.16:20:37.494>
    closer = 'vstinner'
    components = ['Interpreter Core']
    creation = <Date 2019-03-11.13:53:27.342>
    creator = 'cstratak'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 36262
    keywords = ['patch']
    message_count = 8.0
    messages = ['337668', '337669', '337670', '337690', '337863', '337943', '337945', '337946']
    nosy_count = 3.0
    nosy_names = ['mark.dickinson', 'vstinner', 'cstratak']
    pr_nums = ['12276', '12331', '12332']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue36262'
    versions = ['Python 2.7', 'Python 3.7', 'Python 3.8']

    @stratakis
    Copy link
    Mannequin Author

    stratakis mannequin commented Mar 11, 2019

    Coverity report on dtoa.c. It was run on python2 but the same code resides on python3.

    Error: RESOURCE_LEAK (CWE-772): [#def89]
    Python-2.7.15/Python/dtoa.c:1846: alloc_fn: Storage is returned from allocation function "s2b".
    Python-2.7.15/Python/dtoa.c:526:9: alloc_fn: Storage is returned from allocation function "multadd".
    Python-2.7.15/Python/dtoa.c:479:13: alloc_fn: Storage is returned from allocation function "Balloc".
    Python-2.7.15/Python/dtoa.c:371:13: alloc_fn: Storage is returned from allocation function "PyMem_Malloc".
    Python-2.7.15/Objects/object.c:2348:5: alloc_fn: Storage is returned from allocation function "malloc".
    Python-2.7.15/Objects/object.c:2348:5: return_alloc_fn: Directly returning storage allocated by "malloc".
    Python-2.7.15/Python/dtoa.c:371:13: var_assign: Assigning: "rv" = "PyMem_Malloc(len * 8UL)".
    Python-2.7.15/Python/dtoa.c:379:5: return_alloc: Returning allocated memory "rv".
    Python-2.7.15/Python/dtoa.c:479:13: var_assign: Assigning: "b1" = "Balloc(b->k + 1)".
    Python-2.7.15/Python/dtoa.c:486:13: var_assign: Assigning: "b" = "b1".
    Python-2.7.15/Python/dtoa.c:491:5: return_alloc: Returning allocated memory "b".
    Python-2.7.15/Python/dtoa.c:526:9: var_assign: Assigning: "b" = "multadd(b, 10, *s++ - 48)".
    Python-2.7.15/Python/dtoa.c:530:5: return_alloc: Returning allocated memory "b".
    Python-2.7.15/Python/dtoa.c:1846: var_assign: Assigning: "bd0" = storage returned from "s2b(s0, nd0, nd, y)".
    Python-2.7.15/Python/dtoa.c:2249: leaked_storage: Variable "bd0" going out of scope leaks the storage it points to.
    2247|
    2248| undfl:
    2249|-> return sign ? -0.0 : 0.0;
    2250|
    2251| ovfl:

    Error: RESOURCE_LEAK (CWE-772): [#def90]
    Python-2.7.15/Python/dtoa.c:2006: alloc_fn: Storage is returned from allocation function "diff".
    Python-2.7.15/Python/dtoa.c:952:5: alloc_fn: Storage is returned from allocation function "Balloc".
    Python-2.7.15/Python/dtoa.c:371:13: alloc_fn: Storage is returned from allocation function "PyMem_Malloc".
    Python-2.7.15/Objects/object.c:2348:5: alloc_fn: Storage is returned from allocation function "malloc".
    Python-2.7.15/Objects/object.c:2348:5: return_alloc_fn: Directly returning storage allocated by "malloc".
    Python-2.7.15/Python/dtoa.c:371:13: var_assign: Assigning: "rv" = "PyMem_Malloc(len * 8UL)".
    Python-2.7.15/Python/dtoa.c:379:5: return_alloc: Returning allocated memory "rv".
    Python-2.7.15/Python/dtoa.c:952:5: var_assign: Assigning: "c" = "Balloc(a->k)".
    Python-2.7.15/Python/dtoa.c:962:5: var_assign: Assigning: "xc" = "c".
    Python-2.7.15/Python/dtoa.c:996:5: return_alloc: Returning allocated memory "c".
    Python-2.7.15/Python/dtoa.c:2006: var_assign: Assigning: "delta" = storage returned from "diff(bb, bd)".
    Python-2.7.15/Python/dtoa.c:2016: noescape: Resource "delta" is not freed or pointed-to in "cmp".
    Python-2.7.15/Python/dtoa.c:890:13: noescape: "cmp(Bigint *, Bigint *)" does not free or save its parameter "a".
    Python-2.7.15/Python/dtoa.c:2129: noescape: Resource "delta" is not freed or pointed-to in "ratio".
    Python-2.7.15/Python/dtoa.c:1179:15: noescape: "ratio(Bigint *, Bigint *)" does not free or save its parameter "a".
    Python-2.7.15/Python/dtoa.c:2249: leaked_storage: Variable "delta" going out of scope leaks the storage it points to.
    2247|
    2248| undfl:
    2249|-> return sign ? -0.0 : 0.0;
    2250|
    2251| ovfl:

    @stratakis stratakis mannequin added 3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Mar 11, 2019
    @vstinner
    Copy link
    Member

    ... in practice, it seems like Python became the "upstream". I see lot of changes, but I'm not sure that version maintained by David M. Gay on http://www.netlib.org/fp/dtoa.c has been updated since Mark Dickinson copied it to Python/dtoa.c:

    commit b08a53a
    Author: Mark Dickinson <dickinsm@gmail.com>
    Date: Thu Apr 16 19:52:09 2009 +0000

    Issue bpo-1580: use short float repr where possible.
     - incorporate and adapt David Gay's dtoa and strtod
       into the Python core
     - on platforms where we can use Gay's code (almost
       all!), repr(float) is based on the shortest
       sequence of decimal digits that rounds correctly.
     - add sys.float_repr_style attribute to indicate
       whether we're using Gay's code or not
     - add autoconf magic to detect and enable SSE2
       instructions on x86/gcc
     - slight change to repr and str:  repr switches
       to exponential notation at 1e16 instead of
       1e17, str switches at 1e11 instead of 1e12
    

    @vstinner
    Copy link
    Member

    Julia copied the same file. See:

    @mdickinson
    Copy link
    Member

    ... in practice, it seems like Python became the "upstream".

    Yes; unfortunately, we changed things enough that updating from upstream became impractical. At some point we should take a look at changes made to the upstream dtoa.c since our adoption of it, and figure out whether any of those changes need to be applied to our copy. That's not going to be an easy task. It would be easier if there were upstream testcases (and regression tests in particular), but as far as I'm aware there aren't any.

    @vstinner
    Copy link
    Member

    New changeset 9776b06 by Victor Stinner in branch 'master':
    bpo-36262: Fix _Py_dg_strtod() memory leak (goto undfl) (GH-12276)
    9776b06

    @vstinner
    Copy link
    Member

    New changeset 9818360 by Victor Stinner in branch '3.7':
    bpo-36262: Fix _Py_dg_strtod() memory leak (goto undfl) (GH-12276) (GH-12331)
    9818360

    @vstinner
    Copy link
    Member

    New changeset b140578 by Victor Stinner in branch '2.7':
    bpo-36262: Fix _Py_dg_strtod() memory leak (goto undfl) (GH-12276) (GH-12332)
    b140578

    @vstinner
    Copy link
    Member

    Thanks for the report Charalampos. I fixed dtoa.c in 2.7, 3.7 and master branches.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants