Thanks for clarifying. :)

On Mon, Feb 18, 2019 at 1:17 PM Stefan Behnel <report@bugs.python.org> wrote:
> The exception fields are performance critical, and we try hard to make them visible to the C compiler so that swapping around exception state eats up as little CPU time as possible.
>
> You could argue that profiling and tracing are less critical, but any nanosecond that is avoided while not tracing a function adds up to making the rest of the program faster, so I'd argue that that's performance critical, too. Profiling definitely is, because it should have as little impact on the code profile as possible. There is a huge difference between having the CPU pre-fetch a pointer and looking at the value, compared to calling into a C function and guessing what the result might be.

Yeah, I had a feeling that was the case. :)  There might be decent
approaches to avoiding a performance hit on this, but I'm not sure
it's worth the extra complexity.  And I'm not terribly motivated to
make the effort personally. :)

> The trashcan is only used during deallocation, so … well, I guess it could be replaced by a different API, but that's a bit tricky due to the bracket nature of the current macros.

Yep, definitely tricky.

> I also just noticed that "Py_EnterRecursiveCall" and "Py_LeaveRecursiveCall" are on your list. We use them in our inlined call helper functions, which mostly duplicate CPython functionality. Looking at these macros now, I find it a bit annoying that they call "PyThreadState_GET()" directly, rather than accepting one as input. Looking up the current thread-state is a non-local, atomic operation that can be surprisingly costly, and I've invested quite some work into reducing these lookups in Cython. Although it's probably not too bad around a call into an external function…

Hmm, perhaps that's something we could address separately?

> Could you explain what benefit you are expecting from hiding the thread state?

This isn't so much about the thread state specifically or about
necessarily hiding *all* the thread state.  Rather it's about a
general effort to reduce the amount of private API we are exposing
publicly.  In this case I was looking at PyInterpreterState (#35886)
and opening a similar issue at the same time for PyThreadState made
sense.

Given your response it's clear that usage of PyThreadState doesn't
match the documentation very well.  That would be worth addressing one
way or another, so I'm glad we're having this conversation. :)