Author decaz
Recipients asvetlov, decaz, yselivanov
Date 2018-03-21.15:48:58
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1521647338.78.0.467229070634.issue33115@psf.upfronthosting.co.za>
In-reply-to
Content
I want to schedule a lot of parallel tasks, but it becomes slow with loop blocking:

```python
import asyncio

task_count = 10000

async def main():
    for x in range(1, task_count + 1):
        asyncio.ensure_future(f(x))

async def f(x):
    if x % 1000 == 0 or x == task_count:
        print(f'Run f({x})')
    await asyncio.sleep(1)
    loop.call_later(1, lambda: asyncio.ensure_future(f(x)))

loop = asyncio.get_event_loop()
loop.set_debug(True)
loop.run_until_complete(main())
loop.run_forever()
```

Outputs:
```
Executing <Task finished coro=<main() done, defined at test_aio.py:5> result=None created at /usr/lib/python3.6/asyncio/base_events.py:446> took 0.939 seconds

...

Executing <TimerHandle when=1841384.785427177 _set_result_unless_cancelled(<Future finis...events.py:275>, None) at /usr/lib/python3.6/asyncio/futures.py:339 created at /usr/lib/python3.6/asyncio/tasks.py:480> took 0.113 seconds

...

Executing <Task pending coro=<f() running at test_aio.py:12> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fe89344fcd8>()] created at /usr/lib/python3.6/asyncio/base_events.py:275> created at test_aio.py:13> took 0.100 seconds

...
```

What can be another way to schedule a lot of parallel tasks?
History
Date User Action Args
2018-03-21 15:48:58decazsetrecipients: + decaz, asvetlov, yselivanov
2018-03-21 15:48:58decazsetmessageid: <1521647338.78.0.467229070634.issue33115@psf.upfronthosting.co.za>
2018-03-21 15:48:58decazlinkissue33115 messages
2018-03-21 15:48:58decazcreate