Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Captured exceptions are keeping user objects alive unnecessarily in the stdlib #81001

Closed
mariocj89 mannequin opened this issue May 6, 2019 · 4 comments
Closed

Captured exceptions are keeping user objects alive unnecessarily in the stdlib #81001

mariocj89 mannequin opened this issue May 6, 2019 · 4 comments
Labels
3.8 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@mariocj89
Copy link
Mannequin

mariocj89 mannequin commented May 6, 2019

BPO 36820
Nosy @Yhg1s, @warsaw, @ericvw, @mariocj89, @pablogsal, @miss-islington
PRs
  • bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py #13135
  • [3.7] bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135) #17485
  • [3.8] bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135) #17486
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-12-06.14:49:38.004>
    created_at = <Date 2019-05-06.19:32:15.480>
    labels = ['3.8', 'type-feature', 'library']
    title = 'Captured exceptions are keeping user objects alive unnecessarily in the stdlib'
    updated_at = <Date 2019-12-06.15:01:35.177>
    user = 'https://github.com/mariocj89'

    bugs.python.org fields:

    activity = <Date 2019-12-06.15:01:35.177>
    actor = 'miss-islington'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-12-06.14:49:38.004>
    closer = 'pablogsal'
    components = ['Library (Lib)']
    creation = <Date 2019-05-06.19:32:15.480>
    creator = 'mariocj89'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 36820
    keywords = ['patch']
    message_count = 4.0
    messages = ['341620', '357916', '357917', '357918']
    nosy_count = 6.0
    nosy_names = ['twouters', 'barry', 'ericvw', 'mariocj89', 'pablogsal', 'miss-islington']
    pr_nums = ['13135', '17485', '17486']
    priority = 'low'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue36820'
    versions = ['Python 3.8']

    @mariocj89
    Copy link
    Mannequin Author

    mariocj89 mannequin commented May 6, 2019

    There are multiple places in the standard library where the code captures an exception and reraises it later (outside of the original except).

    This is known to cause a cycle as saving the exception has a traceback that eventually points back to the exception, but it is especially problematic as it keeps alive the objects that the user has as well.

    This can be reproduced with the following example:

    import weakref
    
    class A: pass
    
    ref = None
    
    def x():
        global ref
        cool_var = A()
        ref = weakref.ref(cool_var)
        assert ref()
        try:
            1/0
        except Exception as e:
            ee = e
    
    try:
        x()
    except Exception:
        pass
    
    print(ref())
    assert ref() is None
    

    There are places in the standard library that try to get around this. See acb9fa7 as an example. This change did not include the exception path though, when the err variable is raised, the issue is still present.

    This issue is to fix the instances found in socket.py, codeop.py and dyld.py. By either setting the variable to None to remove the name or if it needs to be reraised by using a finally to delete it. This is basically the same that a try except would do, which automagically adds a finally to delete the local name used in the except X as y.
    There was a discussion with twouters,pablogsal,barry about potentially adding something extra to the exception handling to try to clean this up but it was suggested the best approach is for the user to handle it and maybe a linter to potentially flag it, as there might also be multiple references to the exception being captured. It was also proposed to just change the situations where this happens in the standard library to remove the name if possible.

    Note that eventually those objects will just be collected by the gc, it is just an improvement.

    I'm preparing a PR for those three modules.

    @mariocj89 mariocj89 mannequin added 3.8 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels May 6, 2019
    @miss-islington
    Copy link
    Contributor

    New changeset b64334c by Miss Islington (bot) (Mario Corchero) in branch 'master':
    bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135)
    b64334c

    @miss-islington
    Copy link
    Contributor

    New changeset 681285d by Miss Islington (bot) in branch '3.8':
    bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135)
    681285d

    @miss-islington
    Copy link
    Contributor

    New changeset 5ba591f by Miss Islington (bot) in branch '3.7':
    bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135)
    5ba591f

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants