This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Async Call-Stack Reconstruction
Type: enhancement Stage:
Components: asyncio, Interpreter Core Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, dino.viehland, gvanrossum, itamaro, kumaraditya, mpage, yselivanov
Priority: normal Keywords:

Created on 2022-03-01 22:13 by mpage, last changed 2022-04-11 14:59 by admin.

Messages (5)
msg414301 - (view) Author: Matt Page (mpage) * Date: 2022-03-01 22:13
Profiling tools that use the call-stack (i.e. all of them) paint an incomplete picture of what’s really going on in async-heavy codebases. They can only show the stack of the currently executing task; they miss the chain of awaitables that are transitively waiting on the current task. To remedy this, we have added support in Cinder to expose the async call-stack. This consists of the call stack for the currently executing task, followed by the chain of awaitables that are transitively reachable from the currently executing task. See below for a clarifying example.

```
async def f1():
  return await f2()

async def f2():
  return await asyncio.ensure_future(f3())

async def f3():
  return await f4()

async def f4():
  await asyncio.sleep(10)
  return 42
```

When retrieved from f4, the two different stacks (top-of-stack last) are:
sync - [f3, f4]
async - [f1, f2, f3, f4] 


We’d like to merge our implementation into CPython so that other heavy users of asyncio can benefit. This will consist of a few parts:

1. A new limited C-API to set and retrieve the “awaiter” of an awaitable object.
2. Additions to PyAsyncMethods to store function pointers for setting and retrieving the awaiter on instances.
3. An API in managed code to retrieve the async call stack as a list of fully qualified names (i.e. <module>:<class>.<function>).
msg414884 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-03-11 01:15
> We’d like to merge our implementation into CPython

Could you provide a link first, please?
msg414930 - (view) Author: Matt Page (mpage) * Date: 2022-03-11 19:02
Sorry for the confusion, I'm working on a PR. I filed the BPO to gauge interest in the feature.
msg414948 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2022-03-11 23:36
I've recently dabbled a bit in some new primitives for asyncio, and based on that experience I think this would be very useful.

IIRC Trio does this (presumably at considerable cost) in userland.
msg414985 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-03-12 09:30
The idea looks interesting.
The devil in the details I guess.
I'm curious what is the memory and performance penalty.
Waiting for the PR as the discussion starting point.
History
Date User Action Args
2022-04-11 14:59:56adminsetgithub: 91048
2022-03-12 09:30:23asvetlovsetmessages: + msg414985
2022-03-11 23:36:27gvanrossumsetmessages: + msg414948
2022-03-11 19:02:59mpagesetmessages: + msg414930
2022-03-11 11:11:39kumaradityasetnosy: + gvanrossum, kumaraditya
2022-03-11 01:15:58asvetlovsetmessages: + msg414884
2022-03-01 22:13:38mpagecreate