New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
'async with' somehow suppresses unawaited coroutine warnings #76884
Comments
Example (minimal version of python-trio/trio#425): ----- async def open_file():
pass
async def main():
async with open_file(): # Should be 'async with await open_file()'
pass
coro = main()
coro.send(None) Here we accidentally left out an 'await' on the call to 'open_file', so the 'async with' tries to look up 'CoroutineType.__aexit__', which obviously doesn't exist, and the program crashes with an AttributeError("__aexit__"). Yet weirdly, this doesn't trigger a warning about 'open_file' being unawaited. It should! Yury's theory: maybe BEFORE_ASYNC_WITH's error-handling path is forgetting to DECREF the object. |
Nope, that doesn't seem to be it. This version prints "refcount: 2" twice, *and* prints a proper "was never awaited" warning: ----- import sys
async def open_file():
pass
async def main():
open_file_coro = open_file()
print("refcount:", sys.getrefcount(open_file_coro))
print("refcount:", sys.getrefcount(open_file_coro))
coro = main()
try:
coro.send(None)
except:
pass |
Looking at the ceval code, I think Yury's theory is plausible, and we may also be leaving the interpreter's internal stack in a dubious state. Things then get cleaned up if you wrap the async with in a try/except or try/finally: ============== >>> async def try_main():
... try:
... async with open_file():
... pass
... finally:
... pass
...
>>> try_main().send(None)
sys:1: RuntimeWarning: coroutine 'open_file' was never awaited
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in try_main
AttributeError: __aexit__ ============== Unfortunately for that theory, adding braces and "Py_DECREF(POP());" to the relevant error handling branch *doesn't* fix the problem. I also found another way to provoke similar misbehaviour without async with: ========== >>> async def open_file():
... pass
...
>>> open_file()
<coroutine object open_file at 0x7f92fe19c548>
>>> _
<coroutine object open_file at 0x7f92fe19c548>
>>> 1
__main__:1: RuntimeWarning: coroutine 'open_file' was never awaited
1
>>> open_file()
<coroutine object open_file at 0x7f92fe19c548>
>>> del _
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name '_' is not defined
>>> _
<coroutine object open_file at 0x7f92fe19c548>
>>> 1
1
>>> _
1
>>> ========== |
Changing async def main():
async with open_file():
pass to async def main():
c = open_file()
async with c:
pass also makes it print the warning :) Also I've made a test out of this snippet and running tests in refleak mode shows that there's indeed no refleak here. |
The difference between these two functions is two extra opcodes: STORE_FAST/LOAD_FAST before BEFORE_ASYNC_WITH. With them we have a warning. |
So refactoring it into "c = open_file(); async with c" just prolongs the life of the coroutine so that it's GCed outside of WITH_CLEANUP_START/WITH_CLEANUP_FINISH block. Something weird is going on in that block. |
Ah, we should never really get to WITH_CLEANUP_START; the exception should be raised in BEFORE_ASYNC_WITH |
So the problem was that _PyGen_Finalize wasn't issuing any warnings if there's any error set in the current tstate. And in Nathaniel's case, the current error was an AttributeError('__aexit__'). This check is weird, because right before raising the warning, we call PyErr_Fetch to temporarily reset the current exception if any, specifically to raise the warning :) The PR just removes the check. Unless I'm missing something this should fix the issue. |
Well, I feel silly then: bpo-32605 |
Ah, and in my REPL example, the NameError was pending when the internal result storage was getting set back to None. I'm not sure I even knew the "Don't complain when an exception is pending" check existed, so it would have taken me a long time to find that. |
I knew about that "if", but it never fully registered to me why it's there and what it is protecting us from ;) So I had to debug half of ceval loop before I stumbled upon it again ;) |
Merged; closing this issue. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: