classification
Title: Reference cycles from a WeakKeyDictionary value to its key aren’t collected
Type: resource usage Stage:
Components: Library (Lib) Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: andersk
Priority: normal Keywords:

Created on 2021-07-19 21:23 by andersk, last changed 2021-07-19 23:06 by andersk.

Messages (2)
msg397841 - (view) Author: Anders Kaseorg (andersk) * Date: 2021-07-19 21:23
Because WeakKeyDictionary unconditionally maintains strong references to its values, the garbage collector fails to collect a reference cycle from a WeakKeyDictionary value to its key.  For example, the following program unexpectedly leaks memory:

from weakref import WeakKeyDictionary
class C: pass
d = WeakKeyDictionary()
while True:
    c = C()
    d[c] = [c]

I would expect a WeakKeyDictionary value to be marked live _if_ its key is marked live, not unconditionally.  This could be implemented with garbage collector support for ephemerons (https://www.researchgate.net/publication/221320677_Ephemerons_A_New_Finalization_Mechanism).

To motivate this issue, a typical use of WeakKeyDictionary is as a hygienic replacement for patching extra properties into third-party objects:

# before:
obj._extra_state = ExtraState(obj)
# after:
extra_states = WeakKeyDictionary()
extra_states[o] = ExtraState(obj)

However, such a conversion will introduce this memory leak if ExtraState(obj) has any transitive references to obj.

This leak does not occur in JavaScript:

class C {}
const d = new WeakMap();
while (true) {
  const c = new C();
  d[c] = [c];
}
msg397845 - (view) Author: Anders Kaseorg (andersk) * Date: 2021-07-19 23:06
> extra_states[o] = ExtraState(obj)

(Typo for extra_states[obj] = ExtraState(obj), obviously.)
History
Date User Action Args
2021-07-19 23:06:04andersksetmessages: + msg397845
2021-07-19 21:23:17anderskcreate