classification
Title: StreamReader.readexactly() raises GeneratorExit on ProactorEventLoop
Type: behavior Stage:
Components: asyncio Versions: Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, matthew, salgado, twisteroid ambassador, yselivanov
Priority: normal Keywords:

Created on 2019-12-21 16:04 by twisteroid ambassador, last changed 2020-11-29 00:12 by matthew.

Files
File name Uploaded Description Edit
prettysocks.py twisteroid ambassador, 2020-10-20 17:41
error_log_on_linux_python38.txt twisteroid ambassador, 2020-10-21 11:03
Messages (5)
msg358774 - (view) Author: twisteroid ambassador (twisteroid ambassador) * Date: 2019-12-21 16:04
I have been getting these strange exception since Python 3.8 on my Windows 10 machine. The external symptoms are many errors like "RuntimeError: aclose(): asynchronous generator is already running" and "Task was destroyed but it is pending!".

By adding try..except..logging around my code, I found that my StreamReaders would raise GeneratorExit on readexactly(). Digging deeper, it seems like the following line in StreamReader._wait_for_data():

await self._waiter

would raise a GeneratorExit.

There are only two other methods on StreamReader that actually does anything to _waiter, set_exception() and _wakeup_waiter(), but neither of these methods were called before GeneratorExit is raised. In fact, both these methods sets self._waiter to None, so normally after _wait_for_data() does "await self._waiter", self._waiter is None. However, after GeneratorExit is raised, I can see that self._waiter is not None. So it seems the GeneratorExit came from nowhere.

I have not been able to reproduce this behavior in other code. This is with Python 3.8.1 on latest Windows 10 1909, using ProactorEventLoop. I don't remember seeing this ever on Python 3.7.
msg378931 - (view) Author: twisteroid ambassador (twisteroid ambassador) * Date: 2020-10-19 08:07
This problem still exists on Python 3.9 and latest Windows 10.

I tried to catch the GeneratorExit and turn it into a normal Exception, and things only got weirder from here. Often several lines later another await statement would raise another GeneratorExit, such as writer.write() or even asyncio.sleep(). Doesn't matter whether I catch the additional GeneratorExit or not, once code exits this coroutine a RuntimeError('coroutine ignored GeneratorExit') is raised. And it doesn't matter what I do with this RuntimeError, the outermost coroutine's Task always generates an 'asyncio Task was destroyed but it is pending!' error message.

Taking a step back from this specific problem. Does a "casual" user of asyncio need to worry about handling GeneratorExits? Can I assume that I should not see GeneratorExits in user code?
msg379149 - (view) Author: twisteroid ambassador (twisteroid ambassador) * Date: 2020-10-20 17:41
I have attached a script that should be able to reproduces this problem. It's not a minimal reproduction, but hopefully easy enough to trigger.

The script is a SOCKS5 proxy server listening on localhost:1080. In its current form it does not need any external dependencies. Run it on Windows 10 + Python 3.9, set a browser to use the proxy server, and browse a little bit, it should soon start printing mysterious errors involving GeneratorExit.
msg379205 - (view) Author: twisteroid ambassador (twisteroid ambassador) * Date: 2020-10-21 11:03
Well this is unexpected, the same code running on Linux is throwing GeneratorExit-related mysterious exceptions as well. I'm not sure whether this is the same problem, but this one has a clearer traceback. I will attach the full error log, but the most pertinent part seems to be this:


During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/contextlib.py", line 662, in __aexit__
    cb_suppress = await cb(*exc_details)
  File "/usr/lib/python3.8/contextlib.py", line 189, in __aexit__
    await self.gen.athrow(typ, value, traceback)
  File "/opt/prettysocks/prettysocks.py", line 332, in closing_writer
    await writer.wait_closed()
  File "/usr/lib/python3.8/asyncio/streams.py", line 376, in wait_closed
    await self._protocol._get_close_waiter(self)
RuntimeError: cannot reuse already awaited coroutine


closing_writer() is an async context manager that calls close() and await wait_closed() on the given StreamWriter. So it looks like wait_closed() can occasionally reuse a coroutine?
msg382027 - (view) Author: Matthew (matthew) Date: 2020-11-29 00:12
Let me preface this by declaring that I am very new to Python async so it is very possible that I am missing something seemingly obvious. That being said, I've been looking at various resources to try to understand the internals of asyncio and it hasn't led to any insights on this problem thus far.
-----------------

This all sounds quite similar to an experience I am dealing with. I'm working with pub sub within aioredis which internally uses a StreamReader with a function equivalent to readexactly. This all started from debugging "Task was destroyed but it is pending!" to which attempted fixes led to multiple "RuntimeError: aclose(): asynchronous generator is already running" errors.

I did the same thing, adding try excepts everywhere in my code to understand what was happening and this led me to identifying that a regular async function would raise GeneratorExit during await. However, even if I suppress this, the caller awaiting on this function would also raise a GeneratorExit. Suppressing this exception at the top level leads to an unsuspecting (to me) error "coroutine ignored GeneratorExit".

I understand that GeneratorExit is raised in unfinished generators when garbage collected to handle cleanup. And I understand that async functions are essentially a generator in the sense that they yield when they await. So, if the entire coroutine were garbage collected this might trigger GeneratorExit in each nested coroutine. However, from all of my logging I am sure that prior to the GeneratorExit, nothing returns  upwards so there should still be valid references to every object.

I'll include some errors below, in case they may be of relevance:

=== Exception in await of inner async function ===
Traceback (most recent call last):
  File ".../site-packages/uvicorn/protocols/http/httptools_impl.py", line 165, in data_received
    self.parser.feed_data(data)
  File "httptools/parser/parser.pyx", line 196, in httptools.parser.parser.HttpParser.feed_data
httptools.parser.errors.HttpParserUpgrade: 858

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../my_code.py", line 199, in wait_for_update
    return await self.waiter.wait_for_value()
GeneratorExit

=== Exception when suppressing GeneratorExit on the top level ===
Exception ignored in: <coroutine object parent_async_function at 0x0b...>
Traceback (most recent call last):
  File ".../site-packages/websockets/protocol.py", line 229, in __init__
    self.reader = asyncio.StreamReader(limit=read_limit // 2, loop=loop)
RuntimeError: coroutine ignored GeneratorExit
History
Date User Action Args
2020-11-29 00:12:06matthewsetnosy: + matthew
messages: + msg382027
2020-10-21 11:03:28twisteroid ambassadorsetfiles: + error_log_on_linux_python38.txt

messages: + msg379205
2020-10-20 17:41:55twisteroid ambassadorsetfiles: + prettysocks.py

messages: + msg379149
2020-10-19 08:15:17twisteroid ambassadorsetversions: + Python 3.9
2020-10-19 08:07:48twisteroid ambassadorsetmessages: + msg378931
2020-08-26 19:47:29salgadosetnosy: + salgado
2019-12-21 16:04:36twisteroid ambassadorcreate