Looking at the ceval code, I think Yury's theory is plausible, and we may also be leaving the interpreter's internal stack in a dubious state. Things then get cleaned up if you wrap the async with in a try/except or try/finally:

==============
>>> async def try_main():
...     try:
...         async with open_file():
...             pass
...     finally:
...         pass
... 
>>> try_main().send(None)
sys:1: RuntimeWarning: coroutine 'open_file' was never awaited
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in try_main
AttributeError: __aexit__
==============

Unfortunately for that theory, adding braces and "Py_DECREF(POP());" to the relevant error handling branch *doesn't* fix the problem.

I also found another way to provoke similar misbehaviour without async with:

==========
>>> async def open_file():
...     pass
... 
>>> open_file()
<coroutine object open_file at 0x7f92fe19c548>
>>> _
<coroutine object open_file at 0x7f92fe19c548>
>>> 1
__main__:1: RuntimeWarning: coroutine 'open_file' was never awaited
1
>>> open_file()
<coroutine object open_file at 0x7f92fe19c548>
>>> del _
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name '_' is not defined
>>> _
<coroutine object open_file at 0x7f92fe19c548>
>>> 1
1
>>> _
1
>>> 
==========