This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Greek0
Recipients BreamoreBoy, Greek0, christian.heimes, docs@python, exarkun, georg.brandl
Date 2014-01-16.15:44:11
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1389887052.13.0.301820792404.issue6128@psf.upfronthosting.co.za>
In-reply-to
Content
I concur that this aspect of writing Python types in C/C++ needs better
documentation. For a new extension writer (like me), it is not obvious when
PyObject_GC_Track/PyObject_GC_UnTrack must be called or not. Similarly, it is
not clear from the current documentation how to ensure that memory is allocated
with PyObject_GC_New.

The gcsupport documentation suffers from being written from the perspective
of the GC, instead of the perspective of extension writers. Thus, I've attached
a patch that tries to shed some light on how to correctly write types
implementing cyclic GC support.

I'd appreciate comments/editing suggestions/advice and would welcome the patch
to go into Python in some form. I myself consider the change to be trivial, but
if necessary I can sign the contributor agreement.

--
PS: Informal explanation about what's really going on:

The Python GC code deals with memory allocation, not with initialization in any
way. For most extension types, allocation is handled by tp_alloc which is
called typically called from tp_new (either the default tp_new or your own
tp_new handler).

The default tp_alloc (PyType_GenericAlloc()) looks at tp_flags, and if
Py_TPFLAGS_HAVE_GC is specified, it uses the appropriate GC malloc function and
calls _PyObject_GC_TRACK when finished). Thus, if you allocate your memory via
tp_alloc, and don't override the tp_alloc handler, you've just fulfilled the
GC's constructor requirements.

Similarly, the GC doesn't care about object destruction. It cares about freeing
memory, that is tp_free, NOT tp_dealloc! Again, if you don't intentionally
provide a custom tp_free handler, the default one will take care to obey the GC
rules.

Thus, most extension writers can get away with 4 simple rules:
* Set Py_TPFLAGS_HAVE_GC in tp_flags.
* Do not specify tp_alloc or tp_free.
* Make sure your object actually allocates it's memory via tp_alloc.
  Three possibilities to achieve this:
    + Do not add a tp_new handler, or
    + Call tp_alloc from your tp_new handler, or
    + Call a base class's tp_new from your own tp_new handler
* Implement tp_traverse and tp_clear. They are usually straight-forward.

And if you really do want a custom allocator, you're screwed anyway, as the
last part of my patch hints at. Custom memory management is just not compatible with the GC at this point.
History
Date User Action Args
2014-01-16 15:44:12Greek0setrecipients: + Greek0, georg.brandl, exarkun, christian.heimes, docs@python, BreamoreBoy
2014-01-16 15:44:12Greek0setmessageid: <1389887052.13.0.301820792404.issue6128@psf.upfronthosting.co.za>
2014-01-16 15:44:12Greek0linkissue6128 messages
2014-01-16 15:44:11Greek0create