Author vstinner
Recipients vstinner
Date 2020-03-06.09:24:50
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1583486690.53.0.82859773355.issue39873@roundup.psfhosted.org>
In-reply-to
Content
One of the worst issue that I had to debug is a crash in the Python garbage collector. It is usually a crash in visit_decref(). See my notes:
https://pythondev.readthedocs.io/debug_tools.html#debug-crash-in-garbage-collection-visit-decref

My previous attempt to help debugging such issue failed: bpo-36389 "Add gc.enable_object_debugger(): detect corrupted Python objects in the GC". The idea was to check frequently if all objects tracked by the GC are valid. The problem is that even if the check looked trivial, checking all objects made Python way slower. Even when I tried to only check objects of the "young" GC generation (generation 0), it was still too slow.

Here I propose a different approach: attempt to only check objects when they are accessed. Recently, I started to replace explicit cast to (PyObject *) type with an indirection: a new _PyObject_CAST() macro which should be the only way to cast any object pointer to (PyObject *).

/* Cast argument to PyObject* type. */
#define _PyObject_CAST(op) ((PyObject*)(op))

This macro is used in many "core" macros like Py_TYPE(op), Py_REFCNT(op), Py_SIZE(op), Py_SETREF(op, op2), Py_VISIT(op), etc.

The idea here is to inject code in _PyObject_CAST(op) when Python is built in debug mode to ensure that the object is valid. The intent is to detect corrupted objects earlier than a garbage collection, to ease debugging C extensions.

The checks should be limited to reduce the performance overhead.

Attached PR implemnts this idea.
History
Date User Action Args
2020-03-06 09:24:50vstinnersetrecipients: + vstinner
2020-03-06 09:24:50vstinnersetmessageid: <1583486690.53.0.82859773355.issue39873@roundup.psfhosted.org>
2020-03-06 09:24:50vstinnerlinkissue39873 messages
2020-03-06 09:24:50vstinnercreate