Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reference leaks introduced by bpo-30860 #75601

Closed
vstinner opened this issue Sep 11, 2017 · 8 comments
Closed

Reference leaks introduced by bpo-30860 #75601

vstinner opened this issue Sep 11, 2017 · 8 comments
Assignees
@ericsnowcurrently
Labels
3.7 (EOL) end of life performance Performance or resource usage

Comments

@vstinner
Copy link
Member

BPO 31420
Nosy @ncoghlan, @vstinner, @ericvsmith, @skrah, @ericsnowcurrently
PRs
  • bpo-30860: Fix a refleak. #3506
  • bpo-30860: Fix a refleak. #3567
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/ericsnowcurrently'
closed_at = <Date 2017-09-12.01:00:56.078>
created_at = <Date 2017-09-11.16:29:21.025>
labels = ['3.7', 'performance']
title = 'Reference leaks introduced by bpo-30860'
updated_at = <Date 2017-09-14.07:36:00.199>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2017-09-14.07:36:00.199>
actor = 'eric.snow'
assignee = 'eric.snow'
closed = True
closed_date = <Date 2017-09-12.01:00:56.078>
closer = 'eric.snow'
components = []
creation = <Date 2017-09-11.16:29:21.025>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 31420
keywords = ['patch']
message_count = 8.0
messages = ['301882', '301883', '301886', '301889', '301891', '301904', '301906', '302154']
nosy_count = 5.0
nosy_names = ['ncoghlan', 'vstinner', 'eric.smith', 'skrah', 'eric.snow']
pr_nums = ['3506', '3567']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue31420'
versions = ['Python 3.7']
    @vstinner
    Copy link
    Member Author

    The following commands show memory leaks introduced by bpo-30860:

    $ ./python -m test -R 3:3 test_atexit -m test.test_atexit.SubinterpreterTest.test_callbacks_leak

$ ./python -m test -R 3:3 test_atexit -m test.test_atexit.SubinterpreterTest.test_callbacks_leak_refcycle

$ ./python -m test -R 3:3 test_threading -m test.test_threading.SubinterpThreadingTests.test_threads_join

$ ./python -m test -R 3:3 test_capi -m test.test_capi.SubinterpreterTest.test_subinterps

    @vstinner vstinner added 3.7 (EOL) end of life performance Performance or resource usage labels Sep 11, 2017
    @vstinner
    Copy link
    Member Author

    The following commands show memory leaks introduced by bpo-30860:

    Oh, I'm talking about the commit 2ebc5ce.

    @skrah
    Copy link
    Mannequin

    skrah mannequin commented Sep 11, 2017

    Could be the same as bpo-31408.

    @vstinner
    Copy link
    Member Author

    The leak comes from the following lines of _PySys_EndInit():
    ---

        PyObject *warnoptions = get_warnoptions();
    if (warnoptions == NULL)
        return -1;
    SET_SYS_FROM_STRING_BORROW_INT_RESULT("warnoptions", warnoptions);

    PyObject *xoptions = get_xoptions();
    if (xoptions == NULL)
        return -1;
    SET_SYS_FROM_STRING_BORROW_INT_RESULT("_xoptions", xoptions);

    It's not the first time that I have an issue with these attributes. The last reference weak caused by multiple interpreters was also related to this one if I recall correctly.

    See bpo-30598 and my commit 865de27.

    @vstinner
    Copy link
    Member Author

    Extract of my msg295399: "The problem is that warnoptions is stored in a C global variable *and* in sys.warnoptions of each interpreter. The ownership of this variable is unclear."

    Maybe we need a change similar to bpo-28411 (commit 86b7afd) which removed the "modules" field from Py_InterpreterState.

    @ericsnowcurrently ericsnowcurrently self-assigned this Sep 11, 2017
    @ericsnowcurrently
    Copy link
    Member

    I'm looking into this.

    @ericsnowcurrently
    Copy link
    Member

    Moving warnoptions (and xoptions) out of PyInterpreterState seems like a good idea to me for the same reasons as applied to sys.modules. The case for these two is even stronger since they are only used in sysmodule.c.

    @ericsnowcurrently
    Copy link
    Member

    New changeset dae0276 by Eric Snow in branch 'master':
    bpo-30860: Fix a refleak. (bpo-3567)
    dae0276

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @ericsnowcurrently ericsnowcurrently

    Labels
    3.7 (EOL) end of life performance Performance or resource usage
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @vstinner @ericsnowcurrently