classification
Title: Make exception wrapping less intrusive for __set_name__ calls
Type: enhancement Stage: patch review
Components: Interpreter Core Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: carljm, ncoghlan, serhiy.storchaka, sir-sigurd
Priority: normal Keywords: patch

Created on 2018-05-19 05:41 by ncoghlan, last changed 2018-06-17 09:03 by sir-sigurd.

Pull Requests
URL Status Linked Edit
PR 6983 closed carljm, 2018-05-19 06:20
Messages (6)
msg317098 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-05-19 05:41
Type creation currently wraps all exceptions raised by __set_name__ calls with a generic RuntimeError: https://github.com/python/cpython/blob/master/Objects/typeobject.c#L7263

Unfortunately, this makes it difficult to use __set_name__ for descriptor validation operations, since it means the specific exception type gets lost, and it makes the traceback much harder to read (since the generic error messages appears at the end, while the actual error appears somewhere in the middle).

See https://bugs.python.org/issue21145#msg317097 for a specific example.
msg317104 - (view) Author: Carl Meyer (carljm) * Date: 2018-05-19 06:36
Nick, I think the reason this exception wrapping was added is because the stack trace for these exceptions is currently a bit lacking. The "caller" for the `__set_name__` function is the `class` line for the class containing the descriptors.

For exceptions raised _inside_ `__set_name__` this is kind of OK, although if a class has multiple instances of the same descriptor class on it, it doesn't give you an obvious clue which instance raised the exception ( though you can probably figure it out quickly enough by checking the value of the `name` argument).

For cases where the exception is raised at the caller (e.g. a TypeError due to a `__set_name__` method with wrong signature) it's worse; you get no pointer to either the problematic descriptor instance, or its name, or the class; all you get is the TypeError and the class that contains a broken descriptor.

In practice I don't know how much of a problem this is; it doesn't seem likely that it would take too long to narrow down the source of the issue.

Let me know what you think.
msg317115 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-05-19 12:40
Hmm, I wonder if the UX problem with the current chaining might be solved in a different way, by doing something similar to what we did for codec exceptions (where we want to try to mention the codec name, but also don't want to change the exception type, and want to include the original exception text in the wrapper's message).

The codecs related call is at https://github.com/python/cpython/blob/0c1c4563a65ac451021d927058e4f25013934eb2/Python/codecs.c#L389 but most of the heavy lifting has since been refactored out into the _PyErr_TrySetFromCause helper function: https://github.com/python/cpython/blob/55edd0c185ad2d895b5d73e47d67049bc156b654/Objects/exceptions.c#L2713
msg317123 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-05-19 14:54
Unconditional replacing an exception is bad, because it can hide important exceptions like KeybordInterrupt or MemoryError.

What if raise a new exception only if TypeError was raised? This will cover the case of a __set_name__() method with wrong signature.
msg317126 - (view) Author: Carl Meyer (carljm) * Date: 2018-05-19 15:47
Awkwardly, the motivating use case in issue21145 is a TypeError that we wanted to raise within __set_name__, and not have replaced. It feels a little ugly to special case TypeError this way. 

I like the _PyErr_TrySetFromCause idea. That function is a bit ugly too, in the way it has to try and sniff out whether an exception has extra state or is safe to copy and add extra context to. But in practice I think the results would be pretty good here. Most of the time you’d get the original exception but with added useful context; occasionally for some exception types you might just not get the extra context. But as long as TypeError falls in the former category it would help with the worst case. 

I’ll look at using that in the PR.
msg317143 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-05-19 23:40
Yeah, the "transparent exception chaining" code falls into the category of code that I'm both proud of (since it works really well in typical cases [1]) and somewhat horrified by (since the *way* it works tramples over several principles of good object oriented design and is completely non-portable to other Python implementations).

Anyway, I've adjusted the issue title here to indicate that we don't want to remove the exception wrapping entirely, just make it less intrusive.

[1] See the hex encoding and bz2 decoding failure examples in https://docs.python.org/3/whatsnew/3.4.html#codec-handling-improvements
History
Date User Action Args
2018-06-17 09:03:07sir-sigurdsetnosy: + sir-sigurd
2018-05-19 23:40:57ncoghlansetmessages: + msg317143
title: Remove exception wrapping from __set_name__ calls -> Make exception wrapping less intrusive for __set_name__ calls
2018-05-19 15:47:22carljmsetmessages: + msg317126
2018-05-19 14:54:14serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg317123
2018-05-19 12:40:14ncoghlansetmessages: + msg317115
2018-05-19 06:36:11carljmsetnosy: + carljm
messages: + msg317104
2018-05-19 06:20:24carljmsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request6637
2018-05-19 05:41:19ncoghlancreate