classification
Title: unittest: on failure, TestCase.run() keeps a reference to the exception
Type: Stage: resolved
Components: Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: ezio.melotti, haypo, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2013-12-03 22:46 by haypo, last changed 2013-12-09 00:56 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
unittest_leak.py haypo, 2013-12-03 22:46
unittest_workaround.patch haypo, 2013-12-03 22:46 review
contextmanager_leak.py haypo, 2013-12-03 22:48
contextmanager_leak2.py haypo, 2013-12-03 23:11
generator_workaround.patch haypo, 2013-12-03 23:11 review
frame_ref_cycle.py haypo, 2013-12-03 23:34
unittest_leak.patch haypo, 2013-12-04 00:15 review
Messages (7)
msg205167 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-12-03 22:46
Test attached unittest_leak.py script: you will see MyException.ninstance counter increased up to 10, whereas I expect that MyException is destroyed at TestCase.run() exit.

It looks like a tricky reference cycle between:

- frames
- exc_info local variable of _Outcome.testPartExecutor() context manager
- _Outcome.errors list
- _Outclass instance

Attached unittest_workaround.patch patch works around the issue.
msg205168 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-12-03 22:48
contextmanager_leak.py: shorter script to demonstrate the issue.

Replacing "exc_info = sys.exc_info()" with "sys.exc_info()" works around the issue.
msg205173 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-12-03 23:11
contextmanager_leak2.py: even simpler example, storing a current frame in a local variable of the frame is enough.

generator_workaround.patch is another workaround: call frame.clear() when at generator exit to explicitly break the reference cycle.
msg205174 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-12-03 23:34
frame_ref_cycle.py: if I understood correctly, unittest_leak.py can be simplified to this script. A frame contains a local variable which contains the frame: reference cycle.
msg205177 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-12-03 23:51
I found this issue while working the memory limit feature of tracemalloc module (issue #19817). I opened #19835 to workaround an unlimited loop on PyErr_NoMemory() when Python is out of memory.

See also the issue #17807 and the PEP 442 for a similar reference cycle with generators.
msg205179 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-12-04 00:15
unittest_leak.patch: Fix the the initial bug, unittest_leak.py.

I don't think that it's possible to write a generic fix for frame_ref_cycle.py. Storing sys.exc_info() to format it as a traceback later is a common pattern. Clearing a frame at function exit breaks this use case.
msg205630 - (view) Author: Roundup Robot (python-dev) Date: 2013-12-09 00:56
New changeset 09658ea0b93d by Victor Stinner in branch 'default':
Close #19880: Fix a reference leak in unittest.TestCase. Explicitly break
http://hg.python.org/cpython/rev/09658ea0b93d
History
Date User Action Args
2013-12-09 00:56:40python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg205630

resolution: fixed
stage: resolved
2013-12-04 00:15:10hayposetfiles: + unittest_leak.patch
nosy: + ezio.melotti
messages: + msg205179

2013-12-03 23:51:45hayposetmessages: + msg205177
2013-12-03 23:34:27hayposetfiles: + frame_ref_cycle.py

messages: + msg205174
2013-12-03 23:11:22hayposetfiles: + generator_workaround.patch
2013-12-03 23:11:16hayposetfiles: + contextmanager_leak2.py

messages: + msg205173
2013-12-03 22:48:47hayposetfiles: + contextmanager_leak.py

messages: + msg205168
2013-12-03 22:46:48hayposetfiles: + unittest_workaround.patch
keywords: + patch
2013-12-03 22:46:40haypocreate