This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author njs
Recipients Mark.Shannon, gvanrossum, iritkatriel, ncoghlan, njs, yselivanov
Date 2021-10-04.06:42:55
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1633329776.11.0.630918001556.issue45361@roundup.psfhosted.org>
In-reply-to
Content
Inside 'except' and 'finally' blocks, the interpreter keeps track of the 'active exception in the thread-state. It can be introspected via `sys.exc_info()`, it enables bare `raise`, and it triggers implicit context propagation.

In an odd bit of synchronicity, I recently bumped into two completely separate cases where Python code would benefit from being able to control this state directly:

- in the flat exception groups discussion [1], I noted that it would be useful if concurrency libraries could propagate this state into child tasks

- in bpo-27089, regarding some gnarly code in contextlib.ExitStack that simulates a set of nested try/finally blocks by running callback with the wrong exception context, and then introspecting the result and trying to fix it up after the fact. It turns out this code is difficult to understand and subtly buggy -- the PR at gh-27089 fixes one of the bugs here, but there are others [2]. It would be much simpler if it could just set the correct `exc_info` before calling each cleanup function, and then no fixups would be needed things would be set up correctly in the first place.

[1] https://discuss.python.org/t/flat-exception-groups-alternative-to-pep-654/10433

[2] https://github.com/python/cpython/pull/27089#issuecomment-932892687

But, currently, the *only* way to make an exception 'active' like this is to raise it and then catch it again. And this has some significant limitations:

- It unconditionally mutates the exception -- in particular both __context__ and __traceback__ are modified

- The "active exception" has type Optional[BaseException], since try/raise/except is block syntax, and you can't `raise None`. So if you want to propagate an arbitrary `sys.exc_info()` into a child task or into a simulated `finally` block, then you need two separate code paths depending on whether `sys.exc_info()[1]` is None or not.

- technically I think you can work around both of these issues with enough effort... but since try/raise/except is block syntax, the workarounds can't be hidden inside a function; you have to inline them into each usage site.

So... I'm thinking maybe we should have some stupid-simple, sharp-edged API to set `sys.exc_info()` from Python. Like, idk, `sys.set_exc_info(...)`, as a trivial wrapper around `PyErr_Restore`.

My main uncertainty is that I know the code for handling the exception state is quite complex, between the "exception stack", the tricky optimizations around partially-initialized exc_info tuples, pushing/popping the exc state in ceval.c, the tricks generators/coroutines use to save/restore exc_info across yields, etc. There might be some hidden dragons in here that require careful handling, and maybe it'll turn out we need to push/pop an exception instead of just setting it, or something like that.

But does the core idea make sense? Anyone aware of any dragons off the top of your head?
History
Date User Action Args
2021-10-04 06:42:56njssetrecipients: + njs, gvanrossum, ncoghlan, Mark.Shannon, yselivanov, iritkatriel
2021-10-04 06:42:56njssetmessageid: <1633329776.11.0.630918001556.issue45361@roundup.psfhosted.org>
2021-10-04 06:42:56njslinkissue45361 messages
2021-10-04 06:42:55njscreate