This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Counter-intuitive behavior of Server.close() / wait_closed()
Type: behavior Stage:
Components: asyncio Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, aymeric.augustin, chris.jerdonek, yselivanov
Priority: normal Keywords:

Created on 2018-09-30 14:16 by aymeric.augustin, last changed 2022-04-11 14:59 by admin.

Messages (3)
msg326725 - (view) Author: Aymeric Augustin (aymeric.augustin) * Date: 2018-09-30 14:16
**Summary**

1. Is it correct for `Server.wait_closed()` (as implemented in asyncio) to be a no-op after `Server.close()`?
2. How can I tell that all incoming connections have been received by `connection_made()` after `Server.close()`?

**Details**

After calling `Server.close()`, `_sockets is None`, which makes `Server.wait_closed()` a no-op: it returns immediately without doing anything (as mentioned in https://bugs.python.org/issue33727).

I'm not sure why the docs suggest to call `wait_closed()` after `close()` if it's a no-op. My best guess is: "this design supports third-party event loops that requires an asynchronous API for closing servers, but the built-in event loops don't need that". Does someone know?

I wrote a very simple server that merely accepts connections. I ran experiments where I saturate the server with incoming client connections and close it. I checked what happens around `close()` (and `wait_closed()` -- but as it doesn't do anything after `close()` I'll just say `close()` from now on.)

The current implementation appears to work as documented, assuming an rather low level interpretation of the docs of `Server.close()`.

> Stop serving: close listening sockets and set the sockets attribute to None.

Correct -- I'm not seeing any `accept` calls in `BaseSelectorEventLoop._accept_connection` after `close()`.

> The sockets that represent existing incoming client connections are left open.

Correct -- if "existing incoming client connections" is interpreted as "client connections that have gone through `accept`".

> The server is closed asynchronously, use the wait_closed() coroutine to wait until the server is closed.
 
I'm seeing calls to `connection_made()` _after_ `close()` because `BaseSelectorEventLoop._accept_connection2` triggers `connection_made()` asynchronously with `call_soon()`.

This is surprising for someone approaching asyncio from the public API rather than the internal implementation. `connection_made()` is the first contact with new connections. The concept of "an existing incoming client connection for which `connection_made()` wasn't called yet" is unexpected.

This has practical consequences.

Consider a server that keeps track of established connections via `connection_made` and `connection_lost`. If this server calls `Server.close()`, awaits `Server.wait_closed()`, makes a list of established connections and terminates them, there's no guarantee that all connections will be closed. Indeed, new connections may appear and call `connection_made()` after `close()` and `wait_closed()` returned!

`wait_closed()` seems ineffective for this use case.
msg326732 - (view) Author: Aymeric Augustin (aymeric.augustin) * Date: 2018-09-30 16:35
For now I will use the following hack:

    server.close()
    await server.wait_closed()

    # Workaround for wait_closed() not quite doing
    # what I want.
    await asyncio.sleep(0)

    # I believe that all accepted connections have reached
    # connection_made() at this point.
msg394772 - (view) Author: Aymeric Augustin (aymeric.augustin) * Date: 2021-05-30 16:58
Would it make sense to add `await asyncio.sleep(0)` in `Server.wait_closed()` to ensure that all connections reach `connection_made()` before `wait_closed()` returns?

This would be fragile but it would be an improvement over the current behavior, wouldn't it?
History
Date User Action Args
2022-04-11 14:59:06adminsetgithub: 79033
2021-05-30 16:58:41aymeric.augustinsetmessages: + msg394772
2020-05-23 22:06:33chris.jerdoneksetnosy: + chris.jerdonek
2018-09-30 16:35:38aymeric.augustinsetmessages: + msg326732
2018-09-30 14:16:34aymeric.augustincreate