Title: local varible referenced a Exception won't be collected in function
Type: resource usage Stage: resolved
Components: Interpreter Core Versions: Python 3.7
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: remi.lapeyre, terry.reedy, wangjie
Priority: normal Keywords:

Created on 2020-02-28 11:18 by wangjie, last changed 2020-05-21 12:01 by remi.lapeyre. This issue is now closed.

Messages (2)
msg362873 - (view) Author: Wang Jie (wangjie) Date: 2020-02-28 11:18
I referenced an Exception object in a function and found memory usage will increase constantly in the accident. I think it may be a bug.

I wrote a minimal code to reproduce it.

from threading import local, Thread
from time import sleep

l = {}

def t0():
  b = l.get('e') # memory usage won't increase if I remove this line
    raise Exception('1')
  except Exception as e:
    l['e'] = e

def target():
  while True:


# t = Thread(target=target)
# t.daemon = True
# t.start()

I tried to execute it in IPython and got the following output:

In [1]: run py/

In [2]: import objgraph

In [3]: objgraph.show_growth(limit=3)
frame        78792    +78792
Exception    78779    +78779
traceback    78779    +78779

In [4]: objgraph.show_growth(limit=3)
Exception   100862    +22083
traceback   100862    +22083
frame       100875    +22083

In [5]: objgraph.show_growth(limit=3)
Exception   115963    +15101
traceback   115963    +15101
frame       115976    +15101

And I tried to execute this code in python2.7 and pypy. Both of them won't occur this problem.
msg362955 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-02-29 01:26
For beginners, 'is this a Python bug' questions should usually be directed elsewhere for initial review.

"When an exception has been assigned using as target, it is cleared at the end of the except clause. This is as if

except E as N:

was translated to

except E as N:
        del N

This means the exception must be assigned to a different name to be able to refer to it after the except clause. Exceptions are cleared because with the traceback attached to them, they form a reference cycle with the stack frame, keeping all locals in that frame alive until the next garbage collection occurs."

"l['e']" is the first new name.  "b" in each locals is the second.  Without "b = ..." there is no increase because l['e'] gets replaced on each call.  But inserting each exception into the next locals (see "keeping all locals in that frame alive") in effect makes a linked list of frames and locals.

If one wants to keep multiple exceptions around, best to copy just the info one want kept.
Date User Action Args
2020-05-21 12:01:18remi.lapeyresetpull_requests: - pull_request19562
2020-05-21 11:52:13remi.lapeyresetnosy: + remi.lapeyre

pull_requests: + pull_request19562
2020-02-29 01:26:00terry.reedysetstatus: open -> closed

nosy: + terry.reedy
messages: + msg362955

resolution: not a bug
stage: resolved
2020-02-28 11:18:30wangjiecreate