This is doable, the feature that decides which line is shown in the traceback is the `lnotab` structure in the relevant code object, the structure is described here (https://github.com/python/cpython/blob/3.8/Objects/lnotab_notes.txt) and it supports negative line offsets so it'd be possible to get the with statement closing statements to backup the line numbers to point to the with statement in question.

I have attached a python script which actually achieves this. `correct_with_cleanup_traceback` when passed a function changes the relevant line numbers so the traceback actually point to the with statement when the error is thrown in the __exit__, most of the relevant work is just dealing with the weird format of lnotab.

This is the error thrown by the test script, it has 2 cascading failures in exit so that I could confirm it worked for nested withs and it absolutely points to the relevant with statement as I have highlighted on the traceback,

    Traceback (most recent call last):
      File "/Users/tadhgmcdonald-jensen/Documents/test.py", line 71, in my_test
    >>  with Test(True) as will_fail_first:                             <<<--------- HERE
      File "/Users/tadhgmcdonald-jensen/Documents/test.py", line 67, in __exit__
        raise Exception("error in exit")
    Exception: error in exit

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/Users/tadhgmcdonald-jensen/Documents/test.py", line 78, in <module>
        my_test()
      File "/Users/tadhgmcdonald-jensen/Documents/test.py", line 70, in my_test
    >>  with Test(True) as fail_during_handling:                        <<<--------- HERE 
      File "/Users/tadhgmcdonald-jensen/Documents/test.py", line 67, in __exit__
        raise Exception("error in exit")
    Exception: error in exit