classification
Title: Captured exceptions are keeping user objects alive unnecessarily in the stdlib
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: barry, ericvw, mariocj89, miss-islington, pablogsal, twouters
Priority: low Keywords: patch

Created on 2019-05-06 19:32 by mariocj89, last changed 2019-12-06 15:01 by miss-islington. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 13135 merged mariocj89, 2019-05-06 19:37
PR 17485 merged miss-islington, 2019-12-06 14:43
PR 17486 merged miss-islington, 2019-12-06 14:43
Messages (4)
msg341620 - (view) Author: Mario Corchero (mariocj89) * (Python triager) Date: 2019-05-06 19:32
There are multiple places in the standard library where the code captures an exception and reraises it later (outside of the original except).

This is known to cause a cycle as saving the exception has a traceback that eventually points back to the exception, but it is especially problematic as it keeps alive the objects that the user has as well.

This can be reproduced with the following example:

```
import weakref

class A: pass

ref = None

def x():
    global ref
    cool_var = A()
    ref = weakref.ref(cool_var)
    assert ref()
    try:
        1/0
    except Exception as e:
        ee = e

try:
    x()
except Exception:
    pass

print(ref())
assert ref() is None
```

There are places in the standard library that try to get around this. See https://github.com/python/cpython/commit/acb9fa79fa6453c2bbe3ccfc9cad2837feb90093 as an example. This change did not include the exception path though, when the `err` variable is raised, the issue is still present.

This issue is to fix the instances found in socket.py, codeop.py and dyld.py. By either setting the variable to None to remove the name or if it needs to be reraised by using a finally to delete it. This is basically the same that a try except would do, which automagically adds a finally to delete the local name used in the `except X as y`.
There was a discussion with twouters,pablogsal,barry about potentially adding something extra to the exception handling to try to clean this up but it was suggested the best approach is for the user to handle it and maybe a linter to potentially flag it, as there might also be multiple references to the exception being captured. It was also proposed to just change the situations where this happens in the standard library to remove the name if possible.

Note that eventually those objects will just be collected by the gc, it is just an improvement.

I'm preparing a PR for those three modules.
msg357916 - (view) Author: miss-islington (miss-islington) Date: 2019-12-06 14:27
New changeset b64334cb93d0ddbb551c8cd712942bab2fc72772 by Miss Islington (bot) (Mario Corchero) in branch 'master':
bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135)
https://github.com/python/cpython/commit/b64334cb93d0ddbb551c8cd712942bab2fc72772
msg357917 - (view) Author: miss-islington (miss-islington) Date: 2019-12-06 14:59
New changeset 681285d052977e3a3a82ef665e788946fca1ac59 by Miss Islington (bot) in branch '3.8':
bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135)
https://github.com/python/cpython/commit/681285d052977e3a3a82ef665e788946fca1ac59
msg357918 - (view) Author: miss-islington (miss-islington) Date: 2019-12-06 15:01
New changeset 5ba591fa2c1f74c4a84372fb4ffc0b16863f1ad7 by Miss Islington (bot) in branch '3.7':
bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135)
https://github.com/python/cpython/commit/5ba591fa2c1f74c4a84372fb4ffc0b16863f1ad7
History
Date User Action Args
2019-12-06 15:01:35miss-islingtonsetmessages: + msg357918
2019-12-06 14:59:56miss-islingtonsetmessages: + msg357917
2019-12-06 14:49:38pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-12-06 14:43:24miss-islingtonsetpull_requests: + pull_request16965
2019-12-06 14:43:18miss-islingtonsetpull_requests: + pull_request16964
2019-12-06 14:27:48miss-islingtonsetnosy: + miss-islington
messages: + msg357916
2019-05-06 20:04:26ericvwsetnosy: + ericvw
2019-05-06 19:47:11barrysetnosy: + barry
2019-05-06 19:37:05mariocj89setkeywords: + patch
stage: patch review
pull_requests: + pull_request13048
2019-05-06 19:32:15mariocj89create