classification
Title: OrderedDict ref cycles cause memory leak
Type: resource usage Stage:
Components: Library (Lib) Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: eric.araujo, jek, rhettinger
Priority: low Keywords:

Created on 2010-09-10 19:04 by jek, last changed 2010-09-12 04:23 by rhettinger. This issue is now closed.

Files
File name Uploaded Description Edit
odicttest.py jek, 2010-09-10 19:04 test script
Messages (6)
msg116036 - (view) Author: jason kirtland (jek) Date: 2010-09-10 19:04
Circular graphs of collections.OrderedDict are uncollectable due to the presence of OrderedDict.__del__.

>>> from collections import OrderedDict
>>> import gc
>>> left, right = OrderedDict(), OrderedDict()
>>> left['other'] = right
>>> right['other'] = left
>>> assert not gc.garbage
>>> del left, right
>>> gc.collect()
10
>>> assert not gc.garbage
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError

Not an issue in 3.1.1, have not verified with 3.1.2.
msg116039 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-09-10 19:38
This is an unfortunate aspect of using __del__.
I don't see a way around it without reintroducing
weak references.

Of course, your code can also use weak ref proxies
to avoid creating uncollectible circular garbage.
msg116041 - (view) Author: jason kirtland (jek) Date: 2010-09-10 20:19
I find the behavior surprising compared to dict and other containers, where this is not an issue and weakrefs are not required in user code.  I would not be surprised, however, to have to wait for a gc.collect() to clean up my cycles like I do for regular objects.

For what it's worth, the pure-python alternative linked in the docs uses neither __del__ nor weakrefs, though it is undoubtably less efficient than this implementation.  And the workaround 'del OrderedDict.__del__' seems to work as well, if one is willing to wait for a gc collection.
msg116128 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-09-11 22:44
On #python-dev, Raymond, Amaury and Benjamin agreed that __del__ should be removed, with the rationale that this method should be used to release resources other than memory.
msg116135 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-09-11 23:58
Éric, I've got this one.  Thx.
msg116147 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-09-12 04:22
For 2.7, removed __del__ in r84725.
For 3.2, replaced __del__ with weakrefs in r84727.
History
Date User Action Args
2010-09-12 04:23:00rhettingersetstatus: open -> closed
resolution: fixed
messages: + msg116147
2010-09-11 23:58:42rhettingersetmessages: + msg116135
2010-09-11 22:44:30eric.araujosetnosy: + eric.araujo
messages: + msg116128
2010-09-10 20:19:12jeksetmessages: + msg116041
2010-09-10 19:38:34rhettingersetpriority: normal -> low

messages: + msg116039
2010-09-10 19:10:05benjamin.petersonsetassignee: rhettinger

nosy: + rhettinger
2010-09-10 19:04:02jekcreate