classification
Title: awaiting a wrapped asyncio.Task multiple times gives long, repeative tracebacks
Type: enhancement Stage:
Components: asyncio Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, lilydjwg, yselivanov
Priority: normal Keywords:

Created on 2020-12-19 08:09 by lilydjwg, last changed 2020-12-23 10:40 by asvetlov.

Messages (2)
msg383364 - (view) Author: lilydjwg (lilydjwg) * Date: 2020-12-19 08:09
import asyncio

async def crash(key):
  raise Exception('crash!')

async def wait(fu):
  await fu

async def main():
  crasher = asyncio.create_task(crash(()))
  fs = [wait(crasher) for _ in range(10)]
  for fu in asyncio.as_completed(fs):
    try:
      await fu
    except Exception:
      import traceback
      traceback.print_exc()

if __name__ == '__main__':
  asyncio.run(main())

This code will give a very long traceback 10 times. I expect it to be short.

I'm caching the result of an async function like this:

  async def get(
    self,
    key: Hashable,
    func: Callable[[Hashable], Coroutine[Any, Any, Any]],
  ) -> Any:
    async with self.lock:
      cached = self.cache.get(key)
      if cached is None:
        coro = func(key)
        fu = asyncio.create_task(coro)
        self.cache[key] = fu

    if asyncio.isfuture(cached): # pending
      return await cached # type: ignore
    elif cached is not None: # cached
      return cached
    else: # not cached
      r = await fu
      self.cache[key] = r
      return r

It works fine, except that when there is an exception the traceback is very long.
msg383637 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2020-12-23 10:40
The traceback contains frames from asyncio internal machinery, that's why the traceback is long.

IFIAK Python standard library never filters such calls, asyncio is not an exception.

On the other hand, well-known pytest library supports `__tracebackhide__ = True` for skipping pytest internals in test failure reports.
I'm not sure if we want to do such black magic though.

Plus, sometimes an information about internals is helpful to detect the actual source of the problem,

I have no clear vision of how to skip internal frames easily. 

Yuri, maybe you have an idea?
History
Date User Action Args
2020-12-23 10:40:52asvetlovsetmessages: + msg383637
2020-12-19 08:09:24lilydjwgcreate