classification
Title: threading.local doesn't support cyclic garbage collecting
Type: behavior Stage: resolved
Components: Extension Modules, Library (Lib) Versions: Python 3.2, Python 3.1, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, georg.brandl, gps, pitrou
Priority: normal Keywords: patch

Created on 2008-09-02 12:14 by pitrou, last changed 2010-08-09 22:51 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
threadlocal.patch pitrou, 2010-08-04 00:35
Messages (5)
msg72332 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-09-02 12:14
tp_traverse and tp_clear on threading.local are defined, but the
Py_TPFLAGS_HAVE_GC flag is not set. As a result, cycles are not collected:

>>> import threading, weakref
>>> o = threading.local()
>>> class X(object): pass
... 
>>> x = X()
>>> x.o = o
>>> o.x = x
>>> wr = weakref.ref(x)
>>> del x, o
>>> import gc
>>> gc.collect()
0
>>> wr()
<__main__.X object at 0x9bb0dc4>
msg112341 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2010-08-01 15:03
Do you want to fix that?
msg112685 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-08-03 21:15
This is much more complicated than I thought. The threadstate dict holds a strong reference to each local dict, and therefore the GC will refuse to collect the local dict when there's a cycle.
(when there's no cycle, the local dict is cleared manually by the local object's tp_dealloc)
msg112729 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-08-04 00:13
This is a patch. It modifies the implementation to use intermediate dummy objects and various weakrefs. This allows to break reference cycles even when the thread state dict is still alive (because it isn't involved in the ref cycles anymore). This also has the benefit of fixing another wart, which is still present in the pure Python implementation, though (see "_threading_local keeps the local of the last stopped thread alive" in test_threading_local).

The pure Python implementation should probably be updated, or perhaps removed altogether (since the C version is always compiled anyway).
msg113496 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-08-09 22:51
Committed in r83918 (py3k), r83919 (3.1) and r83920 (2.7) after Benjamin ok'ed it on IRC.
History
Date User Action Args
2010-08-09 22:51:49pitrousetstatus: open -> closed
versions: + Python 3.1, Python 2.7
messages: + msg113496

resolution: fixed
stage: patch review -> resolved
2010-08-04 00:35:49pitrousetkeywords: - easy
files: + threadlocal.patch
2010-08-04 00:35:31pitrousetfiles: - threadlocal.patch
2010-08-04 00:13:56pitrousetfiles: + threadlocal.patch

nosy: + gps
messages: + msg112729

keywords: + patch
stage: patch review
2010-08-03 21:15:21pitrousetversions: - Python 3.1, Python 2.7
2010-08-03 21:15:00pitrousetnosy: + amaury.forgeotdarc
messages: + msg112685
2010-08-01 15:07:23pitrousetkeywords: + easy
assignee: pitrou ->
versions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6, Python 3.0
2010-08-01 15:03:59georg.brandlsetassignee: pitrou

messages: + msg112341
nosy: + georg.brandl
2008-09-02 12:14:04pitroucreate