This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Implement PEP 654: Exception Groups
Type: enhancement Stage: resolved
Components: Documentation, Interpreter Core, Library (Lib) Versions: Python 3.11
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: iritkatriel Nosy List: Mark.Shannon, erlendaasland, gvanrossum, iritkatriel, xtreak
Priority: normal Keywords: patch

Created on 2021-09-26 12:07 by iritkatriel, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 28569 merged iritkatriel, 2021-09-26 14:32
PR 29207 closed iritkatriel, 2021-10-25 10:24
PR 29545 merged iritkatriel, 2021-11-13 14:07
PR 29581 merged iritkatriel, 2021-11-16 19:34
PR 30158 merged iritkatriel, 2021-12-16 22:15
Messages (12)
msg402661 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-09-26 14:37
We will implement Exception Groups and except* in a series of PRs:

1. Add the ExceptionGroup and BaseExceptionGroup classes
2. Update traceback rendering code (python and C) for exception groups.
3. Implement except*
4. Write documentation sections.
msg404837 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-10-22 23:13
New changeset f30ad65dbf3c6b1b5eec14dc954d65ef32327857 by Irit Katriel in branch 'main':
bpo-45292: [PEP 654] add the ExceptionGroup and BaseExceptionGroup classes (GH-28569)
https://github.com/python/cpython/commit/f30ad65dbf3c6b1b5eec14dc954d65ef32327857
msg405773 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-11-05 09:39
New changeset 3509b26c916707363c71a1df040855e395cf4817 by Irit Katriel in branch 'main':
bpo-45292: [PEP 654] Update traceback display code to work with exception groups (GH-29207)
https://github.com/python/cpython/commit/3509b26c916707363c71a1df040855e395cf4817
msg406266 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2021-11-13 04:55
The tests emit some deprecation warnings : 

PYTHONWARNINGS=always ./python -Wall -m test test_exception_group
0:00:00 load avg: 0.39 Run tests sequentially
0:00:00 load avg: 0.39 [1/1] test_exception_group
/home/karthikeyan/stuff/python/cpython/Lib/test/test_exception_group.py:41: DeprecationWarning: invalid escape sequence '\('
  MSG = 'second argument \(exceptions\) must be a sequence'
/home/karthikeyan/stuff/python/cpython/Lib/test/test_exception_group.py:47: DeprecationWarning: invalid escape sequence '\('
  MSG = 'second argument \(exceptions\) must be a non-empty sequence'
/home/karthikeyan/stuff/python/cpython/Lib/test/test_exception_group.py:52: DeprecationWarning: invalid escape sequence '\('
  MSG = ('Item [0-9]+ of second argument \(exceptions\)'

== Tests result: SUCCESS ==

1 test OK.

Total duration: 51 ms
Tests result: SUCCESS
msg406400 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2021-11-16 11:58
New changeset 8b06d01507fd708468570eaa43a349828784489a by Irit Katriel in branch 'main':
bpo-45292: Use raw strings for regex in tests (GH-29545)
https://github.com/python/cpython/commit/8b06d01507fd708468570eaa43a349828784489a
msg408549 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-14 16:48
New changeset d60457a6673cf0263213c2f2be02c633ec2e2038 by Irit Katriel in branch 'main':
bpo-45292: [PEP-654] add except* (GH-29581)
https://github.com/python/cpython/commit/d60457a6673cf0263213c2f2be02c633ec2e2038
msg408557 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-12-14 19:29
We should have a discussion here about improvements that Mark Shannon would like to see, in particular to do more work in the compiler instead of the interpreter.
msg408559 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-12-14 20:21
PR 29581 resulted in a 1% slowdown, which is not terrible, but code not using except* should not be slowed down at all.

IMO, the way to avoid the slowdown is to implement except* using the existing instruction set (perhaps with a few minor additions)

We already implement try-finally, named except blocks and with statements without any complex bytecodes (except perhaps WITH_EXCEPT_START).

These used to involve a lot of state and more complex bytecodes. So it is possible to make these simplifications,
but it does take work.


There are a number of techniques we can use:

If any state is needed, push it to the stack as we do with `ctx.__exit__` in the with statement, and when pushing f_lasti in exception handlers.
Duplicate code paths when the semantics differ in different cases, as we do for finally blocks.
If anything is too complex to handle on the stack, put it in a temporary variable.
Be liberal in your use of virtual try-excepts (SETUP_FINALLY, POP_FINALLY pairs), as zero-cost exception handling should keep the cost down.


It may be too late for this advice, but if I were writing the `except*` implementation from scratch, I would:

1. Sketch out the pseudo Python that a try-except* would map to. This is a good opportunity to discover any design bugs that might result in undesirable behavior for corner cases.

2. Implement the translation in the compiler, not worrying about any redundancy or inefficiency, just correctness.

3. Look to improve the above, either in the compiler front-end, or by replacing inefficient code patterns in the back-end. Handling the optimization in the backend has the advantage that other code might benefit as well.
msg408560 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-14 20:45
The PR adds two new opcodes. Let's start with the simpler of the two - JUMP_IF_NOT_EG_MATCH.   This is the exception-group variation on JUMP_IF_NOT_EXC_MATCH.

JUMP_IF_NOT_EXC_MATCH checks for a match by checking if the exception is of the given type. The result is boolean.

JUMP_IF_NOT_EG_MATCH checks for a matching by calling .split() on the exception group. The result is two exception groups (the matching part and the non-matching part).

Can we do this without a new opcode?
msg408589 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-15 09:34
The second opcode that the PR adds is PREP_RERAISE_STAR.

This opcode takes a list that contains:
1. all the exceptions that were raised in the executed except* clauses
2. the unmatched part of the exception group

It constructs the exception group that needs to be raised at the end.  This is done through a fairly complex operation on the BaseExceptionGroup, which merges the re-raised exceptions into the same nesting structure they had in the original exception group, so that

try:
   raise eg
except* ValueError:
   raise
except* TypeError:
   raise

is equivalent to just 'raise eg'.


Is there any overlap with existing opcodes?
msg408590 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-15 09:39
The way these two opcodes are combined by the compiler to implement except* is described in the pseudo code here: 

https://github.com/python/cpython/pull/29581#issuecomment-975660029

except* uses JUMP_IF_NOT_EG_MATCH. The excepts (not-*) that collect exceptions raised in the except* clauses are virtual.

The do_reraise_star at the end is PREP_RERAISE_STAR followed by POP_EXCEPT_AND_RERAISE.
msg409898 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-06 19:05
New changeset 9925e70e4811841556747a77acd89c1a70bf344a by Irit Katriel in branch 'main':
bpo-45292: [PEP-654] exception groups and except* documentation (GH-30158)
https://github.com/python/cpython/commit/9925e70e4811841556747a77acd89c1a70bf344a
History
Date User Action Args
2022-04-11 14:59:50adminsetgithub: 89455
2022-01-06 19:06:15iritkatrielsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2022-01-06 19:05:44iritkatrielsetmessages: + msg409898
2021-12-16 22:15:59iritkatrielsetpull_requests: + pull_request28377
2021-12-15 09:39:41iritkatrielsetmessages: + msg408590
2021-12-15 09:34:07iritkatrielsetmessages: + msg408589
2021-12-14 20:45:24iritkatrielsetmessages: + msg408560
2021-12-14 20:21:57Mark.Shannonsetmessages: + msg408559
2021-12-14 19:29:54gvanrossumsetnosy: + gvanrossum, Mark.Shannon
messages: + msg408557
2021-12-14 16:48:28iritkatrielsetmessages: + msg408549
2021-11-16 19:34:03iritkatrielsetpull_requests: + pull_request27824
2021-11-16 11:58:31xtreaksetmessages: + msg406400
2021-11-13 14:07:06iritkatrielsetpull_requests: + pull_request27794
2021-11-13 04:55:49xtreaksetnosy: + xtreak
messages: + msg406266
2021-11-05 09:43:42erlendaaslandsetnosy: + erlendaasland
2021-11-05 09:39:50iritkatrielsetmessages: + msg405773
2021-10-25 10:24:56iritkatrielsetpull_requests: + pull_request27475
2021-10-22 23:13:50iritkatrielsetmessages: + msg404837
2021-09-26 21:13:14vstinnersettitle: Implement PEP 654 -> Implement PEP 654: Exception Groups
2021-09-26 14:37:29iritkatrielsetmessages: + msg402661
2021-09-26 14:32:29iritkatrielsetkeywords: + patch
stage: patch review
pull_requests: + pull_request26952
2021-09-26 12:07:11iritkatrielcreate