Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

super.__init__ leaks memory if called multiple times #70905

Closed
KevinModzelewski mannequin opened this issue Apr 9, 2016 · 12 comments
Closed

super.__init__ leaks memory if called multiple times #70905

KevinModzelewski mannequin opened this issue Apr 9, 2016 · 12 comments
Assignees
@serhiy-storchaka
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage

Comments

@KevinModzelewski
Copy link
Mannequin

KevinModzelewski mannequin commented Apr 9, 2016

BPO 26718
Nosy @gvanrossum, @brettcannon, @vstinner, @serhiy-storchaka
Files
  • super_init_leaks.patch
  • super_init_leaks_2.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-04-13.12:38:38.479>
created_at = <Date 2016-04-09.01:02:03.209>
labels = ['interpreter-core', 'performance']
title = 'super.__init__ leaks memory if called multiple times'
updated_at = <Date 2016-04-13.12:38:38.478>
user = 'https://bugs.python.org/KevinModzelewski'

    bugs.python.org fields:

    activity = <Date 2016-04-13.12:38:38.478>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-04-13.12:38:38.479>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2016-04-09.01:02:03.209>
creator = 'Kevin Modzelewski'
dependencies = []
files = ['42445', '42447']
hgrepos = []
issue_num = 26718
keywords = ['patch']
message_count = 12.0
messages = ['263053', '263073', '263103', '263195', '263196', '263232', '263249', '263253', '263254', '263270', '263276', '263331']
nosy_count = 6.0
nosy_names = ['gvanrossum', 'brett.cannon', 'vstinner', 'python-dev', 'serhiy.storchaka', 'Kevin Modzelewski']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue26718'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6']
    @KevinModzelewski
    Copy link
    Mannequin Author

    KevinModzelewski mannequin commented Apr 9, 2016

    The super() __init__ function fills in the fields of a super object without checking if they were already set. If someone happens to call __init__ again, the previously-set references will end up getting forgotten and leak memory.

    For example:

    import sys
print(sys.gettotalrefcount())
sp = super(int, 1)
for i in range(100000):
  super.__init__(sp, float, 1.0)
print(sys.gettotalrefcount())

    @KevinModzelewski KevinModzelewski mannequin added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Apr 9, 2016
    @SilentGhost SilentGhost mannequin added the type-bug An unexpected behavior, bug, or error label Apr 9, 2016
    @serhiy-storchaka
    Copy link
    Member

    If super used __new__ instead of __init__, this issue probably wouldn't arise.

    I'm surprised that super is subclassable.

    @gvanrossum
    Copy link
    Member

    Any uses of super() beyond the documented idioms are uninteresting, except they should not be usable as crash or DoS vectors.

    @brettcannon
    Copy link
    Member

    Based on Guido's feedback and the fact that this isn't documented usage of super() and hence no promises to not re-initialize, I'm closing as "not a bug". Sorry, Kevin.

    @gvanrossum
    Copy link
    Member

    Actually, the refcount bug is still a bug.

    @gvanrossum gvanrossum reopened this Apr 11, 2016
    @serhiy-storchaka
    Copy link
    Member

    Possible solutions:

    1. Correctly decref old values.
    2. Raise an exception if super.__init__ is caled multiple times.
    3. Remove super.__init__ and add super.__new__.

    What is more preferable?

    @gvanrossum
    Copy link
    Member

    Do #1.

    --Guido (mobile)
    On Apr 11, 2016 11:31 PM, "Serhiy Storchaka" <report@bugs.python.org> wrote:

    Serhiy Storchaka added the comment:

    Possible solutions:

    1. Correctly decref old values.
    2. Raise an exception if super.__init__ is caled multiple times.
    3. Remove super.__init__ and add super.__new__.

    What is more preferable?

    ----------

    Python tracker <report@bugs.python.org>
    <http://bugs.python.org/issue26718\>

    @serhiy-storchaka
    Copy link
    Member

    Here is a patch.

    @serhiy-storchaka serhiy-storchaka added performance Performance or resource usage and removed type-bug An unexpected behavior, bug, or error labels Apr 12, 2016
    @vstinner
    Copy link
    Member

    super_init_leaks.patch LGTM, it fixes. I confirm that the patch fixes the refleak. I checked with:

    $ ./python -m test -R 3:3 test_super

    @serhiy-storchaka
    Copy link
    Member

    Added more comments as suggested by Guido.

    @vstinner
    Copy link
    Member

    super_init_leaks_2.patch LGTM.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Apr 13, 2016

    New changeset 450f36750cb9 by Serhiy Storchaka in branch '3.5':
    Issue bpo-26718: super.__init__ no longer leaks memory if called multiple times.
    https://hg.python.org/cpython/rev/450f36750cb9

    New changeset 4680438f486f by Serhiy Storchaka in branch '2.7':
    Issue bpo-26718: super.__init__ no longer leaks memory if called multiple times.
    https://hg.python.org/cpython/rev/4680438f486f

    New changeset 55f4c1f8ca6a by Serhiy Storchaka in branch 'default':
    Issue bpo-26718: super.__init__ no longer leaks memory if called multiple times.
    https://hg.python.org/cpython/rev/55f4c1f8ca6a

    @serhiy-storchaka serhiy-storchaka self-assigned this Apr 13, 2016
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @serhiy-storchaka serhiy-storchaka

    Labels
    interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @brettcannon @vstinner @gvanrossum @serhiy-storchaka