Nathaniel Smith:
> There are other leakable resources besides heap and GPU memory -- shmem segments and file descriptors are two that I thought of earlier, but there are probably others too. (Note that not all file descriptors are associated with a python file object.) I guess one could hack most of these things into pointer bit tricks somehow, but I don't really see the appeal myself :-). It's fine for an internal implementation trick, but a very pythonic public api...

Ok ok, you convinced me. I opened the issue #26588: "_tracemalloc: add support for multiple address spaces (domains)".

Hum, the idea of tracking file descriptors is appealing. In practice, I suggest to simply use Py_uintptr_t type in the new C API rather than void*. So it's more obvious that tracemalloc doesn't care of the value, it only requires the value to be unique.

FYI On Windows, handles and file descriptors are also different namespaces. You can get a handle and a file descriptor which have the same value but are identify different objects.

I don't know shmem.

--

Another different idea would be to annotate a "color" to a memory allocator. For example, group memory only used internally, and memory to store user data. For a network application, you can imagine an identifier per request and then group all allocations of this request.

It sounds very similar to the domain idea, but there is a very important difference: the API proposed in issue #26588 requires to pass domain to track *and* untrack functions. The problem with coloring allocations is that the color is known when the memory is allocated, but the function releasing the memory can be very far, and it can simply be the garbage collector. So the color must be stored in the hashtable *value* (the trace), whereas the domain is stored in the hashtable *key* (a tuple (pointer, domain)).

Since no user directly asked this feature, I propose to defer this issue and focus on the concrete issue #26588 to support tracking GPU(s) memory.