This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Teach pdb to step through asyncio et al.
Type: enhancement Stage:
Components: asyncio Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, giampaolo.rodola, njs, ppperry, smurfix, yselivanov
Priority: normal Keywords: patch

Created on 2018-02-21 21:21 by smurfix, last changed 2022-04-11 14:58 by admin.

Files
File name Uploaded Description Edit
pdb.diff smurfix, 2018-02-21 21:21
Messages (9)
msg312509 - (view) Author: Matthias Urlichs (smurfix) * Date: 2018-02-21 21:21
The attached patch is a proof-of-concept implementation of a way to teach pdb to "single-step" through non-interesting code that you can't skip with "n". The prime example for this is asyncio, trio et al., though decorators like @contextlib.contextmanager also benefit.

A "real" implementation should allow the user to specify ranges to ignore, on the pdb command line (probably by filename and optional range of line numbers, instead of pattern matching). A visual indication of how much code has been skipped-ahead that way might also be beneficial.
msg312510 - (view) Author: Matthias Urlichs (smurfix) * Date: 2018-02-21 21:21
File attachment failed, retrying …
msg312513 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2018-02-21 21:44
Can you back up a bit and give an example of the problem you're running into?
msg312555 - (view) Author: Matthias Urlichs (smurfix) * Date: 2018-02-22 11:49
"Example of a problem"? Well, just single-step into, and then back out of, an @asynccontextmanager-decorated function.

@asynccontextmanager
async def gen():
    yield 1234

async def foo():
    import pdb;pdb.set_trace()
    async with gen() as x:
        assert x == 1234
    print("done")

trio.run(foo)
# asyncio.get_event_loop().run_until_complete(foo())

Let's assume that you want to single step through that context manager. Now count how often you need to press "s" until you arrive at the print statement. With my patch it's 7. Without the patch there are 30 additional steps within the asynccontextmanager. If you need Python 3.6 compatibility and also use @async_generator you need to hit Return ~130 times. That's way too much.

Or: let's say you want to step across an await statement that actually goes through a context switch. You hit "n" and see a <Return>. With something like my patch, and assuming there's no other context to switch to, hitting "s" gets you back to the current context.
msg312560 - (view) Author: Matthias Urlichs (smurfix) * Date: 2018-02-22 12:36
*Sigh*. ... if you need Python 3.5 compatibility ... obviously.
msg312581 - (view) Author: (ppperry) Date: 2018-02-22 18:47
This feature already exists and doesn't need to be reimplemented; use `pdb.Pdb(skip={"trio", "contextlib", ...}).run(...)`, and in any case should be disableable. I don't use asyncio, but happen to have a habit of running pdb through the standard library for no reason and don't want to have that option taken away.
msg312582 - (view) Author: (ppperry) Date: 2018-02-22 18:49
Just to be clear, I'm not opposing having the default value for `skip` be something other than the empty set, but it should be overridable, and the default should include some other things this patch omits, like `importlib._bootstrap(_external)?`
msg312589 - (view) Author: Matthias Urlichs (smurfix) * Date: 2018-02-22 19:26
Ah. Thank you for pointing me to that feature, I completely missed it.

The proposed enhancement thus boils down to "implement a couple of pdb commands to display and modify this skip list". I'm +1 on keeping the default empty.

When you're already in the debugger, mired in the guts of an intractable bug, restarting it all just to set up a skip list isn't sufficient.
msg312779 - (view) Author: Nathaniel Smith (njs) * (Python committer) Date: 2018-02-25 07:56
The help for 'n' says:

(Pdb) help n
n(ext)
        Continue execution until the next line in the current function
        is reached or it returns.

(And the docs [1] say essentially the same.)

It sounds like if that were true, then you wouldn't have a problem, right? But what it's actually doing is continuing until the next line in the current function is reached, or it returns, *or it yields*.

It would be super helpful if pdb had a way to step past yields, running at full speed until the current frame is resumed. And it's not just 'next' where this would be useful, it's useful for 'step' and 'until' as well.

Furthermore, I think we should distinguish between the "user visible" yields in generators and async generators, and the "implementation detail" yields to the coroutine runner that are async functions do. In both cases you might want stepping to follow the yield or not, so it'd be nice to have the option, but the *default* for user-visible yields should be to treat the yield like a return, and the *default* for implementation-detail yields should be to continue until the frame is resumed again.

[1] https://docs.python.org/3.7/library/pdb.html#pdbcommand-next
History
Date User Action Args
2022-04-11 14:58:58adminsetgithub: 77081
2018-02-25 07:56:04njssetmessages: + msg312779
2018-02-22 19:29:48ppperrysetversions: + Python 3.8
2018-02-22 19:26:08smurfixsetmessages: + msg312589
2018-02-22 18:49:26ppperrysetmessages: + msg312582
2018-02-22 18:47:40ppperrysetnosy: + ppperry
messages: + msg312581
2018-02-22 12:36:00smurfixsetmessages: + msg312560
2018-02-22 11:49:38smurfixsetmessages: + msg312555
2018-02-21 21:44:26njssetmessages: + msg312513
2018-02-21 21:21:59smurfixsetfiles: + pdb.diff
keywords: + patch
messages: + msg312510
2018-02-21 21:21:20smurfixcreate