classification
Title: AsyncExitStack.enter_async_context() is mishandling exception __context__
Type: Stage: patch review
Components: Library (Lib) Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: David Hoyes, John Belmonte, jbelmonte, ncoghlan, njs
Priority: normal Keywords: patch

Created on 2021-07-09 23:59 by John Belmonte, last changed 2021-07-12 07:48 by John Belmonte.

Pull Requests
URL Status Linked Edit
PR 27089 open jbelmonte, 2021-07-12 00:22
Messages (5)
msg397230 - (view) Author: John Belmonte (John Belmonte) Date: 2021-07-09 23:59
Over at the Trio project, we have evidence that `AsyncExitStack.enter_async_context(foo())` is not actually equivalent to `async with foo()` regarding raised exception context.

The symptom is a very long, unhelpful tracebacks because the __context__ of raised exceptions is not set to the expected object.

https://github.com/python-trio/trio/issues/2001

I can't speak to this solution myself, but njsmith suggests this amendment to contextlib:

    saved_context = exc_details[1].__context__
        try:
            raise exc_details[1]
        finally:
            exc_details[1].__context__ = saved_context
msg397271 - (view) Author: John Belmonte (John Belmonte) Date: 2021-07-12 00:56
demonstrating the difference for async case:

    import contextlib
    import trio
    
    async def background():
        assert False
    
    async def main1():
        async with trio.open_nursery() as nursery:
            nursery.start_soon(background)
            await trio.sleep_forever()
    
    async def main2():
        async with contextlib.AsyncExitStack() as stack:
            nursery = await stack.enter_async_context(trio.open_nursery())
            nursery.start_soon(background)
            await trio.sleep_forever()
    
    try:
        trio.run(main1)
    except BaseException as e:
        print('main1, context:', e.__context__)
    
    try:
        trio.run(main2)
    except BaseException as e:
        print('main2, context:', e.__context__)


----
main1, context: None
main2, context: Cancelled
msg397284 - (view) Author: John Belmonte (John Belmonte) Date: 2021-07-12 06:31
To clarify the problem case, I believe the discrepancy is seen when raising exceptions as follows:

exc = foo()
try:
    raise exc
finally:
    exc.__context__ = None

(From my understanding, Trio has valid use cases for doing this since it wants to control complex exception chaining, and this is beyond the scope of __suppress_context__.)

Neither ExitStack nor AsyncExcitStack are preserving the None context in the case above.

  === `with` statement ===
  Traceback (most recent call last):
    File "exit_stack_test.py", line 251, in <module>
      assert False
    File "/.../python3.7/contextlib.py", line 130, in __exit__
      self.gen.throw(type, value, traceback)
    File "exit_stack_test.py", line 244, in my_cm
      raise exc
  MyException

  === enter_context() ===
  Traceback (most recent call last):
    File "exit_stack_test.py", line 240, in my_cm
      yield
    File "exit_stack_test.py", line 259, in <module>
      assert False
  AssertionError

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "exit_stack_test.py", line 259, in <module>
      assert False
    File "/.../python3.7/contextlib.py", line 524, in __exit__
      raise exc_details[1]
    File "/.../python3.7/contextlib.py", line 509, in __exit__
      if cb(*exc_details):
    File "/.../python3.7/contextlib.py", line 377, in _exit_wrapper
      return cm_exit(cm, exc_type, exc, tb)
    File "/.../python3.7/contextlib.py", line 130, in __exit__
      self.gen.throw(type, value, traceback)
    File "exit_stack_test.py", line 244, in my_cm
      raise exc
  MyException
msg397285 - (view) Author: John Belmonte (John Belmonte) Date: 2021-07-12 06:33
[reposting the example, with source]

example:
  class MyException(Exception):
      pass

  @contextmanager
  def my_cm():
      try:
          yield
      except BaseException:
          exc = MyException()
          try:
              raise exc
          finally:
              exc.__context__ = None

  print('\n=== `with` statement ===')
  try:
      with my_cm():
          assert False
  except BaseException as e:
      traceback.print_exc()

  print('\n=== enter_context() ===')
  try:
      with ExitStack() as stack:
          stack.enter_context(my_cm())
          assert False
  except BaseException as e:
      traceback.print_exc()


output:
  === `with` statement ===
  Traceback (most recent call last):
    File "exit_stack_test.py", line 251, in <module>
      assert False
    File "/.../python3.7/contextlib.py", line 130, in __exit__
      self.gen.throw(type, value, traceback)
    File "exit_stack_test.py", line 244, in my_cm
      raise exc
  MyException

  === enter_context() ===
  Traceback (most recent call last):
    File "exit_stack_test.py", line 240, in my_cm
      yield
    File "exit_stack_test.py", line 259, in <module>
      assert False
  AssertionError

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "exit_stack_test.py", line 259, in <module>
      assert False
    File "/.../python3.7/contextlib.py", line 524, in __exit__
      raise exc_details[1]
    File "/.../python3.7/contextlib.py", line 509, in __exit__
      if cb(*exc_details):
    File "/.../python3.7/contextlib.py", line 377, in _exit_wrapper
      return cm_exit(cm, exc_type, exc, tb)
    File "/.../python3.7/contextlib.py", line 130, in __exit__
      self.gen.throw(type, value, traceback)
    File "exit_stack_test.py", line 244, in my_cm
      raise exc
  MyException
msg397289 - (view) Author: John Belmonte (John Belmonte) Date: 2021-07-12 07:48
cc: ncoghlan for help with ExitStack exception context
History
Date User Action Args
2021-07-12 07:48:36John Belmontesetnosy: + ncoghlan
messages: + msg397289
2021-07-12 06:33:45John Belmontesetmessages: + msg397285
2021-07-12 06:31:57John Belmontesetmessages: + msg397284
2021-07-12 00:56:20John Belmontesetmessages: + msg397271
2021-07-12 00:22:47jbelmontesetkeywords: + patch
nosy: + jbelmonte

pull_requests: + pull_request25637
stage: patch review
2021-07-10 22:31:24David Hoyessetnosy: + David Hoyes
2021-07-09 23:59:53John Belmontecreate