I'd like to add some argument to providing a "file" and "line number" to the allocation api.  I know that currently this is not provided e.g. by the PyMem_Allocate() functions, but I think it would be wise to provide a "debug" version of these functions that pass in the call sites.  An allocator api that then also allows for these values to be provided to the malloc/realloc/free routines is then future-proof in that respect.

Case in point:  We have a memory profiler running which uses a allocator hook system similar to what Victor is proposing.  But in addition, it provides a "file " and "line" argument to every function.

Now, the profiler is currently not using this code.  Here how the "malloc" function looks:

static void *
PyMalloc(size_t size, void *arg, const char *file, int line, const char *msg)
{
    void *r = DustMalloc(size);
    if (r) {
        tmAllocEx(g_telemetryContext, file, line, r, size, "Python alloc: %s", msg);
		ReportAllocInfo(AllocEvent, 0, r, size);
    }
    return r;
}

tmAllocEx is calling the Telemetry memory profiles and passing in the allocation site.  (http://www.radgametools.com/telemetry.htm, also my blog about using it:  http://cosmicpercolator.com/2012/05/25/optimizing-python-condition-variables-with-telemetry/

But our profiler, called with ReportAllocInfo, isn't using this.  It relies solely on extracting the python callstack.

Today, I got this email (see attached file Capture.jpg)  

Basically, the profiler sees a lot of allocated memory with no python call stack.  Now it would be useful if we had the C call site information, to know where it came from.

So:  My suggestion is that the allocator api be
1) a struct, which allows for a cleaner api function
2) Include C filename and line number.

Even though the current python memory API (e.g. PyMem_Malloc(), PyObject_Malloc()) do not currently support it, this would allow us to internally have _extended_ versions of these apis that do support it and macros that pass in that information.  This can be added at a later stage.  Having it in the allcoator api function would make it more future proof.

See also my "pymem.h" and "ccpmem.h" files attached to this defect for examples on how we have tweaked python's internal memory apis to support this information.