New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use an iterative implementation for contextlib.ExitStack.__exit__ #59168
Comments
The current implementation of contextlib.ExitStack [1] actually creates a nested series of frames when unwinding the callback stack in an effort to ensure exceptions are chained correctly, just as they would be if using nested with statements. It would be nice to avoid this overhead by just using the one frame to iterate over the callbacks and handling correct exception chaining directly. This is likely to be a little tricky to get right, though, so the first step would be to set up a test that throws and suppresses a few exceptions and ensures the chaining when using ExitStack matches that when using nested with statements. [1] http://hg.python.org/cpython/file/94a5bf416e50/Lib/contextlib.py#l227 |
The iterative approach turned out elegant and concise. |
Sorry, I wasn't clear on what I meant by "chained correctly", and that's the part that makes this trickier than the way contextlib.nested did it. I'm referring to the __context__ attribute on exceptions that is set automatically when an exception occurs in another exception handler, which makes error displays like the following possible: >>> from contextlib import ExitStack
>>> with ExitStack() as stack:
... @stack.callback
... def f():
... 1/0
... @stack.callback
... def f():
... {}[1]
...
Traceback (most recent call last):
File "/home/ncoghlan/devel/py3k/Lib/contextlib.py", line 243, in _invoke_next_callback
suppress_exc = _invoke_next_callback(exc_details)
File "/home/ncoghlan/devel/py3k/Lib/contextlib.py", line 240, in _invoke_next_callback
return cb(*exc_details)
File "/home/ncoghlan/devel/py3k/Lib/contextlib.py", line 200, in _exit_wrapper
callback(*args, **kwds)
File "<stdin>", line 7, in f
KeyError: 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 5, in <module>
File "/home/ncoghlan/devel/py3k/Lib/contextlib.py", line 256, in __exit__
return _invoke_next_callback(exc_details)
File "/home/ncoghlan/devel/py3k/Lib/contextlib.py", line 245, in _invoke_next_callback
suppress_exc = cb(*sys.exc_info())
File "/home/ncoghlan/devel/py3k/Lib/contextlib.py", line 200, in _exit_wrapper
callback(*args, **kwds)
File "<stdin>", line 4, in f
ZeroDivisionError: division by zero The recursive approach maintains that behaviour automatically because it really does create a nested set of exception handlers. With the iterative approach, we leave the exception handler before invoking the next callback, so we end up bypassing the native chaining machinery and will need to recreate it manually. If you can make that work, then we'd end up with the best of both worlds: the individual exceptions would be clean (since they wouldn't be cluttered with the recursive call stack created by the unwinding process), but exception chaining would still keep track of things if multiple exceptions are encountered in cleanup operations. |
that was indeed trickier, but overriding the __context__ attribute did the trick. |
New changeset fc73e6ea9e73 by Nick Coghlan in branch 'default': |
New changeset c0c7618762e5 by Nick Coghlan in branch 'default': |
Interesting - it turns out we can't fully reproduce the behaviour of nested with statements in ExitStack (see the new reference test I checked in, as well as bpo-14969) I added one technically redundant variable to the implementation to make it more obviously correct to the reader, as well as a test that ensures the stack can handle ridiculous numbers of callbacks without failing (a key advantage of using a single frame rather than one frame per callback) While it isn't mandatory, we prefer it if contributors submit Contributor Agreements even for small changes. If you're happy to do that, I consider emailing a scanned or digitally photographed copy of the signed form as described here to be the simplest currently available approach: http://www.python.org/psf/contrib/ |
after bpo-14969 has closed, can this be closed? any more action items? |
It *was* closed - I inadvertently reopened it with my comment. Fixed :) |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: