If an object's destructor contains native code which calls PyErr_PrintEx, and that object's refcount drops to zero as the result of an async function returning, the async function incorrectly returns None.

I first identified this behavior while using Boost-python. A more detailed description, and steps to reproduce, are in the issue report I filed on that library: https://github.com/boostorg/python/issues/374

I'm not very familiar with interpreter internals, so it is possible that this is expected behavior. However, it does seem like at least a leaky abstraction between the mechanics of async calls (which use exception based control flow internally) and the PyErr_PrintEx function, which is typically invoked by callers interested in finding out about errors that they caused, not errors that are both caused elsewhere and whose propagation is important to preserving call stack state.

Zac, the documentation for PyErr_Print [1] states that it clears the error indicator. This is akin to handling the exception. Was this your intention?

If you want to print the exception without clearing it, I think you would probably need to call PyErr_Fetch, then PyErr_Display, then PyErr_Restore.

[1] https://docs.python.org/3/c-api/exceptions.html#c.PyErr_PrintEx

Irit, the documentation is clear.

However, the problem is that the presence of StopIteration in the exception global during async return time causes very unexpected behavior.

If, during async return time (in the linked issue, I do this in a boost-python object destructor), I write code like this:

        try {
            boost::python::call(somefunction);
        } catch (boost::python::error_already_set e) {
            PyErr_Print();
        }

then two things go wrong:
1. The "catch" block runs even if "somefunction" did not raise, because StopIteration is already in the exception handler.
2. When PyErr_Print clears StopIteration, the currently-returning async function returns None, regardless of what it was supposed to return. I consider this to be interpreter-induced data corruption.

I'm not sure what happens if user code raises an additional exception.

Do you think this is a Python bug, or a Boost bug? I'm not familiar enough with interpreter internals to know whether or not there are other ways besides boost-python destructors to get code to run "during async return". If there are, I would definitely consider this a Python bug: PyErr_Print should not be able to corrupt interpreter/return state.

For context (copied from the linked GitHub issue), this has affected at least two OSS projects:

https://github.com/apache/pulsar/issues/11823
https://github.com/tbodt/v8py/issues/45

The proposed alternative ("PyErr_Fetch, then PyErr_Display, then PyErr_Restore") does not work well: if code runs in a boost-python destructor which runs during async return time, how would it tell the difference between a "real" exception raised from within whatever function is currently returning and the "fake" StopIteration exception that is in the error global while an async function returns? Aka, this code would not have any way to function properly:

    async def func():
        thing = MyNativeCodeObject()
        raise ValueError()

The destructor for MyNativeCodeObject would have to perform a complex check, and PyErr_Restore/do nothing IFF the following are all true:
- Is there an exception currently being handled/in the global?
- Is that exception a StopIteration?
- Is the interpreter currently in the process of returning from an async function (I'm not sure how to check this)?


The fact that non-StopIteration exceptions are handled correctly (e.g. the ValueError in the above example makes it out of the async call during "await func()") means, I think, that there's already some secondary exception storage mechanism that's used to stash and restore user exceptions while the async machinery raises StopIteration to return from an async function. Could that same mechanism be used to "hide" the StopIteration from user code that runs during async-return time?

'catch (boost::python::error_already_set e)' is equal to `except BaseException as e:`
In Python, blind catching base exception is dangerous, the code should re-raise it usually.
The same is true for boost::python usage.

> how would it tell the difference between a "real" exception raised from within whatever function is currently returning and the "fake" StopIteration exception that is in the error global while an async function returns?

There is no "fake" exception. async function is a kind of Python generator object that uses StopIteration exception for finishing.
The same is true for a regular Python iterator; nothing asyncio specific.

I suggest writing a functional equivalent for `except Exception as e: print(e)` instead of catching BaseException error.