This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Consequences of using Py_TPFLAGS_HAVE_GC are incompletely explained
Type: enhancement Stage: needs patch
Components: Documentation Versions: Python 3.4, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Greek0, christian.heimes, docs@python, exarkun, georg.brandl
Priority: low Keywords: patch

Created on 2009-05-27 20:48 by exarkun, last changed 2022-04-11 14:56 by admin.

Files
File name Uploaded Description Edit
gcsupport-doc.diff Greek0, 2014-01-16 15:44 Proposed doc patch for gcsupport.rst review
Messages (4)
msg88441 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2009-05-27 20:48
Creation of GC'd types is explained at
<http://docs.python.org/c-api/gcsupport.html>.

The docs claim that PyObject_GC_Track must be called once an object
created with PyObject_GC_New is initialized.  The docs fail to explain
what should be done if there is an error during initialization of an
object created with PyObject_GC_New.  Should PyObject_GC_Track still be
called?  Should some other API be called to free the object?

Overall, the docs are missing an explanation of the overall working of
the GC and GC APIs, so it is hard to build an understanding of the
system.  Without an understand, it's difficult to read between the lines
of the API docs, meaning they need to cover every possible case, which
they don't currently.
msg112282 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-08-01 08:10
Can someone in the know provide a doc patch for this?
msg192371 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-07-05 22:57
Python's extension modules aren't consistent. Some places deallocate the object with PyObject_Del(), other places are using PyObject_GC_Del() or simple Py_DECREF().
msg208283 - (view) Author: Christian Aichinger (Greek0) Date: 2014-01-16 15:44
I concur that this aspect of writing Python types in C/C++ needs better
documentation. For a new extension writer (like me), it is not obvious when
PyObject_GC_Track/PyObject_GC_UnTrack must be called or not. Similarly, it is
not clear from the current documentation how to ensure that memory is allocated
with PyObject_GC_New.

The gcsupport documentation suffers from being written from the perspective
of the GC, instead of the perspective of extension writers. Thus, I've attached
a patch that tries to shed some light on how to correctly write types
implementing cyclic GC support.

I'd appreciate comments/editing suggestions/advice and would welcome the patch
to go into Python in some form. I myself consider the change to be trivial, but
if necessary I can sign the contributor agreement.

--
PS: Informal explanation about what's really going on:

The Python GC code deals with memory allocation, not with initialization in any
way. For most extension types, allocation is handled by tp_alloc which is
called typically called from tp_new (either the default tp_new or your own
tp_new handler).

The default tp_alloc (PyType_GenericAlloc()) looks at tp_flags, and if
Py_TPFLAGS_HAVE_GC is specified, it uses the appropriate GC malloc function and
calls _PyObject_GC_TRACK when finished). Thus, if you allocate your memory via
tp_alloc, and don't override the tp_alloc handler, you've just fulfilled the
GC's constructor requirements.

Similarly, the GC doesn't care about object destruction. It cares about freeing
memory, that is tp_free, NOT tp_dealloc! Again, if you don't intentionally
provide a custom tp_free handler, the default one will take care to obey the GC
rules.

Thus, most extension writers can get away with 4 simple rules:
* Set Py_TPFLAGS_HAVE_GC in tp_flags.
* Do not specify tp_alloc or tp_free.
* Make sure your object actually allocates it's memory via tp_alloc.
  Three possibilities to achieve this:
    + Do not add a tp_new handler, or
    + Call tp_alloc from your tp_new handler, or
    + Call a base class's tp_new from your own tp_new handler
* Implement tp_traverse and tp_clear. They are usually straight-forward.

And if you really do want a custom allocator, you're screwed anyway, as the
last part of my patch hints at. Custom memory management is just not compatible with the GC at this point.
History
Date User Action Args
2022-04-11 14:56:49adminsetgithub: 50378
2014-02-03 15:37:37BreamoreBoysetnosy: - BreamoreBoy
2014-01-16 15:44:12Greek0setfiles: + gcsupport-doc.diff
versions: + Python 3.5
nosy: + Greek0

messages: + msg208283

keywords: + patch
2013-07-05 22:57:36christian.heimessetpriority: normal -> low

type: enhancement
versions: + Python 3.4
nosy: + christian.heimes

messages: + msg192371
stage: needs patch
2010-08-01 08:10:42BreamoreBoysetassignee: georg.brandl -> docs@python

messages: + msg112282
nosy: + docs@python, BreamoreBoy
2009-05-27 20:48:54exarkuncreate