This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Trouble subclassing ExceptionGroup
Type: Stage: resolved
Components: Versions: Python 3.11
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, iritkatriel, petr.viktorin, yselivanov
Priority: normal Keywords: patch

Created on 2022-01-19 13:13 by petr.viktorin, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 30852 merged iritkatriel, 2022-01-24 14:42
PR 30854 merged iritkatriel, 2022-01-24 16:56
PR 30901 merged iritkatriel, 2022-01-25 23:18
Messages (15)
msg410942 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2022-01-19 13:13
I want to test a web application by crawling every reachable page. If an error occurs, I need to keep track of the page the error occured at (and additional info like what links were followed to get to the page, so a `__note__` string isn't enough). This info is in an object I'll call Task.
To use the improvements from PEP 654 but also keep extra info, I tried to make a subclass of ExceptionGroup:

```python
class MultiError(ExceptionGroup):
    def __init__(self, failed_tasks):
        super.__init__(
            f"{len(tasks)} tasks failed",
            [t.exception for t in failed_tasks],
        )
        self.tasks = tasks
        # ... and set __note__ on each exception for nice tracebacks
```

but this fails with a rather opaque message:

```python
>>> class Task: exception = AssertionError() # enough of a Task for this example 
... 
>>> MultiError([Task(), Task()])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: function takes exactly 2 arguments (1 given)
```

Knowing about `__new__` and following a note in the docs, I'm able to fix this, but It's not obvious.
Before suggesting stuff, I'll ask: Am I doing something weird, or should this be made easier/more obvious?


Another issue I ran into: the list of exceptions is stored in the publicly-named but undocumented attribute `exceptions`. Am I allowed to use it?
msg410943 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-19 13:38
We probably do need better documentation for subclassing ExceptionGroup.

When you subclass an ExceptionGroup you want to make sure that split() and subgroup() (which are used by except*) will continue working, usually by defining a derive() method:

https://docs.python.org/3.11/library/exceptions.html#BaseExceptionGroup.derive

If you don't define derive the superclass constructor is used, which means you get something of type ExceptionGroup, not your subclass.


I don't know whether it's a good idea to make it easier to define a subclass that doesn't support split()/except* because ti changes the constructor signature without providing derive().
msg410945 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-19 13:40
Re the exceptions attribute - I don't think there's a reason not to document it, I can add that (it is mentioned in the PEP).
msg410946 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-19 13:45
For your use case - can you just assign the task to a field on the exception other than __note__?  The only reason we needed __note__ as an official feature is because we want the interpreter's traceback code to use it. But I think you can assign any field to an exception and then use it in your app:

e.webapp_task = task
msg410947 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2022-01-19 13:49
Thanks for looking into it!

> If you don't define derive the superclass constructor is used, which means you get something of type ExceptionGroup, not your subclass.

That might be fine in my case (for a MVP at least). Is there any other danger in not overriding it?
I see the docs say "A subclass needs to override it", but it might be better to enumerate the perils, or if such a class is unusable, not allow creating it.
msg410948 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2022-01-19 13:51
> can you just assign the task to a field on the exception other than __note__?

That might work, but I'm afraid of touching namespaces I don't own. If the subclass is feasible, I'd rather go with that.
msg410952 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-19 14:24
> Is there any other danger in not overriding it?

No issue as long as you don't use split()/subgroup() or except*.
msg411289 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-01-22 20:26
Is this a matter of adding some documentation showing how it should be done? Even if we don't want to encourage such subclassing, we do have APIs to support it, and I think we ought to show how to do it, given that you have to coordinate a bunch of things.

How feasible is it to improve the error? (If that error is what you always get when __init__ and __new__ don't agree, maybe that could be a separate, more general issue?)
msg411482 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-24 15:42
The error message isn't always this bad:

>>> class Base:
...     def __new__(cls, a, b, c):
...        cls.newargs = (a,b,c)
...     
... 
>>> class Derived(Base):
...     def __init__(self, x):
...         super().__init__(x)
...         self.initargs = (x,)
... 
>>> Derived(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Base.__new__() missing 2 required positional arguments: 'b' and 'c'
msg411493 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-24 16:41
This is the fix:

-    if (!PyArg_ParseTuple(args, "UO", &message, &exceptions)) {
+    if (!PyArg_ParseTuple(args,
+                          "UO:BaseExceptionGroup.__new__",
+                          &message,
+                          &exceptions)) {


Then we get

TypeError: BaseExceptionGroup.__new__() takes exactly 2 arguments (1 given)
msg411495 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-01-24 16:53
So, a PR with that fix would be nice?
msg411519 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-24 21:47
New changeset 573b54515740ce51dcf2402038a9d953aa6c317f by Irit Katriel in branch 'main':
bpo-46431: improve error message on invalid calls to BaseExceptionGroup.__new__ (GH-30854)
https://github.com/python/cpython/commit/573b54515740ce51dcf2402038a9d953aa6c317f
msg411520 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-24 21:50
New changeset b18fd54f8c27e4b2aac222e75ac58aa85e5a7988 by Irit Katriel in branch 'main':
bpo-46431: Add example of subclassing ExceptionGroup. Document the message and exceptions attributes (GH-30852)
https://github.com/python/cpython/commit/b18fd54f8c27e4b2aac222e75ac58aa85e5a7988
msg411521 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-24 21:50
Thank you Petr.
msg411689 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-25 23:53
New changeset 072f4a473e861c6c987650f08990c0ed1f76715f by Irit Katriel in branch 'main':
bpo-46431: use raw string for regex in test (GH-30901)
https://github.com/python/cpython/commit/072f4a473e861c6c987650f08990c0ed1f76715f
History
Date User Action Args
2022-04-11 14:59:54adminsetgithub: 90589
2022-01-25 23:53:07iritkatrielsetmessages: + msg411689
2022-01-25 23:18:17iritkatrielsetpull_requests: + pull_request29080
2022-01-24 21:50:58iritkatrielsetstatus: open -> closed
resolution: fixed
messages: + msg411521

stage: patch review -> resolved
2022-01-24 21:50:27iritkatrielsetmessages: + msg411520
2022-01-24 21:47:59iritkatrielsetmessages: + msg411519
2022-01-24 16:56:32iritkatrielsetpull_requests: + pull_request29035
2022-01-24 16:53:31gvanrossumsetmessages: + msg411495
2022-01-24 16:41:12iritkatrielsetmessages: + msg411493
2022-01-24 15:42:15iritkatrielsetmessages: + msg411482
2022-01-24 14:42:10iritkatrielsetkeywords: + patch
stage: patch review
pull_requests: + pull_request29033
2022-01-22 20:26:58gvanrossumsetmessages: + msg411289
2022-01-19 14:24:34iritkatrielsetnosy: + gvanrossum, yselivanov
2022-01-19 14:24:16iritkatrielsetmessages: + msg410952
2022-01-19 13:52:00petr.viktorinsetmessages: + msg410948
2022-01-19 13:49:16petr.viktorinsetmessages: + msg410947
2022-01-19 13:45:41iritkatrielsetmessages: + msg410946
2022-01-19 13:40:44iritkatrielsetmessages: + msg410945
2022-01-19 13:38:49iritkatrielsetmessages: + msg410943
2022-01-19 13:13:20petr.viktorincreate