When a stack frame belonging to a coroutine that is currently suspended is cleared, via the frame.clear() function, the coroutine appears to stop working properly and hangs forever pending some callback.

I've put up an example program that exhibits this problem here: https://gist.github.com/zerotypic/74ac1a7d6b4b946c14c1ebd86de6027b (and also attached it)

In this example, the stack frame comes from the traceback object associated with an Exception that was raised inside the coroutine, which was passed to another coroutine.

I've included some sample output from the program below. In this run, we do not trigger the bug:

$ python3 async_bug.py
foo: raising exception
foo: waiting for ev
bar: exn = TypeError('blah',)
bar: setting ev
foo: ev set, quitting now.
bar: foo_task = <Task finished coro=<foo() done, defined at async_bug.py:9> result='quit successfully'>

The result is that the task running the coroutine "foo" quits successfully.

In this run, we trigger the bug by providing the "bad" commandline argument:

$ python3 async_bug.py bad
foo: raising exception
foo: waiting for ev
bar: exn = TypeError('blah',)
bar: Clearing frame.
bar: setting ev
bar: foo_task = <Task pending coro=<foo() done, defined at async_bug.py:9> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7faf08aa9378>()]>>
Task was destroyed but it is pending!
task: <Task pending coro=<foo() done, defined at async_bug.py:9> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7faf08aa9378>()]>>


The task running "foo" is still pending, waiting on some callback, even though it should have awakened and completed.

This also happens with generators:

>>> def gen():
...    try:
...       raise TypeError("blah")
...    except Exception as e:
...       yield e
...    print("Completing generator.")
...    yield "done"
... 
>>> gen()
<generator object gen at 0x7f562b6847d8>
>>> list(gen())
Completing generator.
[TypeError('blah',), 'done']
>>> g = gen()
>>> exn = next(g)
>>> exn.__traceback__.tb_frame.clear()
>>> next(g)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> 

This isn't surprising since you shouldn't be able to clear frames of code that is still running. It seems to me that the frame.clear() function should check that the frame belongs to a suspended coroutine/generator, and refuse to clear the frame if so. In frame_clear() (frameobject.c:676), the frame is checked to see if it is currently executing using _PyFrame_IsExecuting(), and raises a RuntimeError if so. Perhaps this should be changed to use _PyFrameHasCompleted()? I am not familiar with the internals of CPython so I'm not sure if this is the right thing to do.

I've been testing this on 3.6.9, but looking at the code for 3.9, this is likely to still be an issue.

Also, I first discovered this due to a call to traceback.clear_frames() from unittest.TestCase.assertRaises(); so if the problem isn't fixed, perhaps unittest should be modified to optionally not clear frames.

Thanks!