classification
Title: Difficult or impossible to figure out how garbage collector and weak references should interact for user-defined extension types
Type: behavior Stage: needs patch
Components: Documentation Versions: Python 3.2, Python 3.3, Python 3.4, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: asvetlov, docs@python, exarkun, jcea, pconnell, pitrou, scoder
Priority: normal Keywords:

Created on 2012-10-11 13:10 by exarkun, last changed 2013-10-05 09:56 by scoder.

Messages (3)
msg172639 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2012-10-11 13:10
There appears to be very little, if any, documentation about how to handle the list at tp_weaklistoffset for types supporting being weak referenced, particularly with respect to garbage collection.

Who owns the list?  Who owns the objects in the list?  Should the list be traversed by the extension type's tp_traverse?  Should it ever be INCREF'd or DECREF'd?  Does tp_dealloc have any responsibilities with respect to it?

From looking at CPython's own source, it appears that tp_traverse should not touch it (Yhg1s confirmed this on #python).  Cython, at least, appears to have come to an alternate conclusion though - it generates tp_traverse functions which visit the list (and provoke gc assertions to fail on Python 2.7 in debug mode).

Some documentation about what is correct to do would be excellent, and would make it easier to explain to Cython developers what they're doing wrong (or confirm that they're doing something right, in which case there are some more bugs to file against the CPython stdlib, since they do something different from Cython).
msg172641 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-10-11 13:22
I think Cython is wrong here. The only thing to do is to call PyObject_ClearWeakRefs() in the deallocator. Everything else is handled by the interpreter.

Agreed improving the documentation would be good.
msg198983 - (view) Author: Stefan Behnel (scoder) * (Python committer) Date: 2013-10-05 09:56
Just as a quick update here: Cython has since then switched to only using PyObject_ClearWeakRefs() and otherwise leaves the handling of the weakref slot to CPython.
History
Date User Action Args
2013-10-05 09:56:16scodersetmessages: + msg198983
2013-10-04 20:05:41pconnellsetnosy: + pconnell
2012-10-19 19:31:35jceasetnosy: + jcea
2012-10-11 13:22:12pitrousetversions: + Python 2.7, Python 3.2, Python 3.3, Python 3.4
nosy: + pitrou, scoder

messages: + msg172641

type: behavior
stage: needs patch
2012-10-11 13:19:07asvetlovsetnosy: + asvetlov
2012-10-11 13:10:21exarkuncreate