awaiting a wrapped asyncio.Task multiple times gives long, repeative tracebacks #86848

lilydjwg · 2020-12-19T08:09:25Z

BPO	42682
Nosy	@asvetlov, @lilydjwg, @1st1, @iritkatriel
Superseder	bpo-45924: Incorrect traceback when future's exception is raised multiple times

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-03-18.09:37:02.741>
created_at = <Date 2020-12-19.08:09:24.657>
labels = ['type-feature', '3.9', 'expert-asyncio']
title = 'awaiting a wrapped asyncio.Task multiple times gives long, repeative tracebacks'
updated_at = <Date 2022-03-18.09:37:02.740>
user = 'https://github.com/lilydjwg'

bugs.python.org fields:

activity = <Date 2022-03-18.09:37:02.740>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-18.09:37:02.741>
closer = 'iritkatriel'
components = ['asyncio']
creation = <Date 2020-12-19.08:09:24.657>
creator = 'lilydjwg'
dependencies = []
files = []
hgrepos = []
issue_num = 42682
keywords = []
message_count = 4.0
messages = ['383364', '383637', '415476', '415477']
nosy_count = 4.0
nosy_names = ['asvetlov', 'lilydjwg', 'yselivanov', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '45924'
type = 'enhancement'
url = 'https://bugs.python.org/issue42682'
versions = ['Python 3.9']

lilydjwg · 2020-12-19T08:09:25Z

import asyncio

async def crash(key):
  raise Exception('crash!')

async def wait(fu):
  await fu

async def main():
  crasher = asyncio.create_task(crash(()))
  fs = [wait(crasher) for _ in range(10)]
  for fu in asyncio.as_completed(fs):
    try:
      await fu
    except Exception:
      import traceback
      traceback.print_exc()

if __name__ == '__main__':
  asyncio.run(main())

This code will give a very long traceback 10 times. I expect it to be short.

I'm caching the result of an async function like this:

  async def get(
    self,
    key: Hashable,
    func: Callable[[Hashable], Coroutine[Any, Any, Any]],
  ) -> Any:
    async with self.lock:
      cached = self.cache.get(key)
      if cached is None:
        coro = func(key)
        fu = asyncio.create_task(coro)
        self.cache[key] = fu

    if asyncio.isfuture(cached): # pending
      return await cached # type: ignore
    elif cached is not None: # cached
      return cached
    else: # not cached
      r = await fu
      self.cache[key] = r
      return r

It works fine, except that when there is an exception the traceback is very long.

asvetlov · 2020-12-23T10:40:52Z

The traceback contains frames from asyncio internal machinery, that's why the traceback is long.

IFIAK Python standard library never filters such calls, asyncio is not an exception.

On the other hand, well-known pytest library supports __tracebackhide__ = True for skipping pytest internals in test failure reports.
I'm not sure if we want to do such black magic though.

Plus, sometimes an information about internals is helpful to detect the actual source of the problem,

I have no clear vision of how to skip internal frames easily.

Yuri, maybe you have an idea?

iritkatriel · 2022-03-18T09:35:37Z

I don't think the OP is complaining about the internal asyncio machinery frames, but rather about the repetition of the "return f.result() # May raise f.exception()." frame.

I will paste the output of the script in the next comment.

iritkatriel · 2022-03-18T09:36:01Z

>>> asyncio.run(main())
Traceback (most recent call last):
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  [Previous line repeated 7 more times]
  File "<stdin>", line 2, in crash
Exception: crash!
Traceback (most recent call last):
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  [Previous line repeated 7 more times]
  File "<stdin>", line 2, in crash
Exception: crash!
Traceback (most recent call last):
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  [Previous line repeated 7 more times]
  File "<stdin>", line 2, in crash
Exception: crash!
Traceback (most recent call last):
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  [Previous line repeated 7 more times]
  File "<stdin>", line 2, in crash
Exception: crash!
Traceback (most recent call last):
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  [Previous line repeated 7 more times]
  File "<stdin>", line 2, in crash
Exception: crash!
Traceback (most recent call last):
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  [Previous line repeated 7 more times]
  File "<stdin>", line 2, in crash
Exception: crash!
Traceback (most recent call last):
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  [Previous line repeated 7 more times]
  File "<stdin>", line 2, in crash
Exception: crash!
Traceback (most recent call last):
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  [Previous line repeated 7 more times]
  File "<stdin>", line 2, in crash
Exception: crash!
Traceback (most recent call last):
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  [Previous line repeated 7 more times]
  File "<stdin>", line 2, in crash
Exception: crash!
Traceback (most recent call last):
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 6, in main
  File "C:\Users\User\src\cpython\Lib\asyncio\tasks.py", line 597, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  File "<stdin>", line 2, in wait
  [Previous line repeated 7 more times]
  File "<stdin>", line 2, in crash
Exception: crash!
>>>

lilydjwg mannequin added 3.9 only security fixes topic-asyncio type-feature A feature request or enhancement labels Dec 19, 2020

iritkatriel closed this as completed Mar 18, 2022

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

awaiting a wrapped asyncio.Task multiple times gives long, repeative tracebacks #86848

awaiting a wrapped asyncio.Task multiple times gives long, repeative tracebacks #86848

lilydjwg mannequin commented Dec 19, 2020

lilydjwg mannequin commented Dec 19, 2020

asvetlov commented Dec 23, 2020

iritkatriel commented Mar 18, 2022

iritkatriel commented Mar 18, 2022

awaiting a wrapped asyncio.Task multiple times gives long, repeative tracebacks #86848

awaiting a wrapped asyncio.Task multiple times gives long, repeative tracebacks #86848

Comments

lilydjwg mannequin commented Dec 19, 2020

lilydjwg mannequin commented Dec 19, 2020

asvetlov commented Dec 23, 2020

iritkatriel commented Mar 18, 2022

iritkatriel commented Mar 18, 2022