➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: asyncio.wait very slow with FIRST_COMPLETED
Type: performance Stage:
Components: asyncio Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, samuelcolvin, yselivanov
Priority: normal Keywords:

Created on 2017-06-02 17:47 by samuelcolvin, last changed 2022-04-11 14:58 by admin.

Files
File name Uploaded Description Edit
asyncio_wait_test.py samuelcolvin, 2017-06-02 17:47
Messages (1)
msg295045 - (view) Author: Samuel Colvin (samuelcolvin) * Date: 2017-06-02 17:47 
This is best described the script at https://gist.github.com/samuelcolvin/00f01793c118bf9aafae886ffbc81a58.

Basically, completing a set of tasks using asyncio.wait(...return_when=asyncio.FIRST_COMPLETED) in a loop to wait for them to finish is much much slower than completing the same set of tasks using just asyncio.wait(pending_tasks, loop=loop, return_when=asyncio.ALL_COMPLETED).

In my example, the ALL_COMPLETED case takes ~5s, but the FIRST_COMPLETED case takes 18 - 47s.

The screenshot on the gist shows network usage with a very long tail, in other words very few tasks are taking a long time to complete.

This example uses aiohttp get a url as the test case but I'm pretty sure this isn't an issue with aiohttp.

I tried the same thing with uvloop but got broadly the same result.
History
Date User Action Args
2022-04-11 14:58:47adminsetgithub: 74741
2022-03-05 06:57:40kumaradityasetnosy: + asvetlov
2017-06-02 17:47:13samuelcolvincreate