classification
Title: Exceptions in asyncio.Server callbacks are not retrievable
Type: Stage:
Components: asyncio Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, tmewett, yselivanov
Priority: normal Keywords:

Created on 2020-12-01 15:42 by tmewett, last changed 2020-12-01 22:33 by asvetlov.

Messages (4)
msg382265 - (view) Author: Tom (tmewett) Date: 2020-12-01 15:42
Consider this program:

    import asyncio

    async def handler(r, w):
        raise RuntimeError

    async def main():
        server = await asyncio.start_server(handler, host='localhost', port=1234)
        r, w = await asyncio.open_connection(host='localhost', port=1234)
        await server.serve_forever()
        server.close()

    asyncio.run(main())

The RuntimeError is not retrievable via the serve_forever coroutine. To my
knowledge, there is no feature of the asyncio API which causes the server to
stop on an exception and retrieve it. I have also tried wrapping serve_forever
in a Task, and waiting on the coro with FIRST_EXCEPTION.

This severely complicates testing asyncio servers, since failing tests hang
forever if the failure occurs in a callback.

It should be possible to configure the server to end if a callback fails, e.g.
by a 'stop_on_error' kwarg to start_server (defaulting to False for
compatibility).

I know this isn't a technical problem, since AnyIO, which uses asyncio, does
this by default. This equivalent program ends after the exception:

    import anyio

    async def handler(client):
        raise RuntimeError

    async def main():
        async with anyio.create_task_group() as tg:
            listener = await anyio.create_tcp_listener(local_host='localhost', local_port=1234)
            await tg.spawn(listener.serve, handler)
            async with await anyio.connect_tcp('localhost', 1234) as client:
                pass

    anyio.run(main)
msg382266 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2020-12-01 15:49
This is a deliberate decision.
An exception in handling one client connection should not break another connected client.
msg382274 - (view) Author: Tom (tmewett) Date: 2020-12-01 17:55
How do you suggest one might test code in a Server callback with asyncio?

Of course, I don't want any old exception to affect another client connection,
only an exception which is uncaught up to the handler coro. And I'm not
suggesting that it happen by default, only that it be possible.

With this, the behaviour would perfectly align with the asyncio.gather
functionality, and its 'return_exceptions' kwarg.
msg382279 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2020-12-01 22:33
You can use try/except in handler() and dispatch the exception whatever you want.
It doesn't require a new asyncio version, debug-only flag, etc.
History
Date User Action Args
2020-12-01 22:33:47asvetlovsetmessages: + msg382279
2020-12-01 17:55:06tmewettsetmessages: + msg382274
2020-12-01 15:49:54asvetlovsetmessages: + msg382266
2020-12-01 15:42:34tmewettcreate