classification
Title: local varible referenced a Exception won't be collected in function
Type: resource usage Stage: resolved
Components: Interpreter Core Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: remi.lapeyre, terry.reedy, wangjie
Priority: normal Keywords:

Created on 2020-02-28 11:18 by wangjie, last changed 2020-05-21 12:01 by remi.lapeyre. This issue is now closed.

Messages (2)
msg362873 - (view) Author: Wang Jie (wangjie) Date: 2020-02-28 11:18
I referenced an Exception object in a function and found memory usage will increase constantly in the accident. I think it may be a bug.

I wrote a minimal code to reproduce it.

```py
from threading import local, Thread
from time import sleep

l = {}

def t0():
  b = l.get('e') # memory usage won't increase if I remove this line
  try:
    raise Exception('1')
  except Exception as e:
    l['e'] = e

def target():
  while True:
    sleep(0.0001)
    t0()

target()

# t = Thread(target=target)
# t.daemon = True
# t.start()
```

I tried to execute it in IPython and got the following output:

```
In [1]: run py/ref_exception_causes_oom.py

In [2]: import objgraph

In [3]: objgraph.show_growth(limit=3)
frame        78792    +78792
Exception    78779    +78779
traceback    78779    +78779

In [4]: objgraph.show_growth(limit=3)
Exception   100862    +22083
traceback   100862    +22083
frame       100875    +22083

In [5]: objgraph.show_growth(limit=3)
Exception   115963    +15101
traceback   115963    +15101
frame       115976    +15101
```

And I tried to execute this code in python2.7 and pypy. Both of them won't occur this problem.
msg362955 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-02-29 01:26
For beginners, 'is this a Python bug' questions should usually be directed elsewhere for initial review.

https://docs.python.org/3/reference/compound_stmts.html#the-try-statement

"When an exception has been assigned using as target, it is cleared at the end of the except clause. This is as if

except E as N:
    foo

was translated to

except E as N:
    try:
        foo
    finally:
        del N

This means the exception must be assigned to a different name to be able to refer to it after the except clause. Exceptions are cleared because with the traceback attached to them, they form a reference cycle with the stack frame, keeping all locals in that frame alive until the next garbage collection occurs."

"l['e']" is the first new name.  "b" in each locals is the second.  Without "b = ..." there is no increase because l['e'] gets replaced on each call.  But inserting each exception into the next locals (see "keeping all locals in that frame alive") in effect makes a linked list of frames and locals.

If one wants to keep multiple exceptions around, best to copy just the info one want kept.
History
Date User Action Args
2020-05-21 12:01:18remi.lapeyresetpull_requests: - pull_request19562
2020-05-21 11:52:13remi.lapeyresetnosy: + remi.lapeyre

pull_requests: + pull_request19562
2020-02-29 01:26:00terry.reedysetstatus: open -> closed

nosy: + terry.reedy
messages: + msg362955

resolution: not a bug
stage: resolved
2020-02-28 11:18:30wangjiecreate