This came up in issue 17468: currently, populating tp_del from C (as generators now do) doesn't automatically create a __del__ wrapper visible from Python.

The rationale given in the initial commit is that there's no need to define a wrapper, since tp_del won't be populated from C code (that will use tp_dealloc instead), but there's now at least one case where it *is* populated from C (generators), which means it behaves *as if* __del__ is defined (since the interpreter actually checks the tp_del slot), but *looks* like __del__ *isn't* defined (since there is no wrapper created).

Independent of the memory leak concerns with generators defining tp_del, it would be better if a wrapper function was defined so the existence of the method was at least visible from Python code.

Would this mean that the destructor could be run more than once (or prematurely)?

Sounds reasonable to me. Note that it won't remove the special-casing in gcmodule.c:

static int
has_finalizer(PyObject *op)
{
    if (PyGen_CheckExact(op))
        return PyGen_NeedsFinalizing((PyGenObject *)op);
    else
        return op->ob_type->tp_del != NULL;
}

What exactly would calling such a wrapper do?

Calling __del__ explicitly shouldn't be any worse than doing the same thing for any other type implemented in Python (or, in the case of generators, calling close() multiple times). What I'm mostly interested in is the "can this type cause uncollectable cycles" introspection aspect.

However, as Antoine noted, generators are an interesting special case because the GC is able to *skip* finalising them in some cases, so exposing __del__ isn't right for them either (as that suggests they will *always* be uncollectable in a cycle, when that isn't the case).

So now I'm wondering if a better answer may be to generalise the current generator special case to a "__needsdel__" protocol: provide a __del__ method, but always make it possible for the GC to skip it when it wouldn't do anything (e.g. if you've already called close() explicitly). PyGenerator_NeedsFinalizing would then become the __needsdel__ impl for generators, and we could lose the special casing in the GC code. From Python, you could detect the three cases through:

__del__ only: can cause uncollectable cycles
__del__and __needsdel__: can cause uncollectable cycles, but it depends on the instance state
Neither: can't cause uncollectable cycles

I don't understand why we need to invent a protocol for this. The gc module already has methods and members for introspecting the collection. I don't think the gen special casing currently needs to be generalized. (What would use it?)

Yeah, I've figured out that rather than exposing __del__ if tp_del is populated, or generalising the generator special case, the simplest way to make this info accessible is to be able to ask the *garbage collector* if it thinks an object needs finalising.

That actually makes this a pretty easy issue (as C issues go) - it's just a matter of exposing http://hg.python.org/cpython/file/default/Modules/gcmodule.c#l525 (has_finalizer) as gc.needs_finalizing.

It will be easier once #17468 is done though, since that will make it clearer what the documentation should say.

Antoine came up with a scheme (see issue 17807) that should eliminate the tp_del implementation from generator objects by moving the cleanup code to the frame deallocation.

This will restore Guido's original assumption that types implemented in C won't need to provide tp_del, so this proposal becomes obsolete.