Message314207
I want to schedule a lot of parallel tasks, but it becomes slow with loop blocking:
```python
import asyncio
task_count = 10000
async def main():
for x in range(1, task_count + 1):
asyncio.ensure_future(f(x))
async def f(x):
if x % 1000 == 0 or x == task_count:
print(f'Run f({x})')
await asyncio.sleep(1)
loop.call_later(1, lambda: asyncio.ensure_future(f(x)))
loop = asyncio.get_event_loop()
loop.set_debug(True)
loop.run_until_complete(main())
loop.run_forever()
```
Outputs:
```
Executing <Task finished coro=<main() done, defined at test_aio.py:5> result=None created at /usr/lib/python3.6/asyncio/base_events.py:446> took 0.939 seconds
...
Executing <TimerHandle when=1841384.785427177 _set_result_unless_cancelled(<Future finis...events.py:275>, None) at /usr/lib/python3.6/asyncio/futures.py:339 created at /usr/lib/python3.6/asyncio/tasks.py:480> took 0.113 seconds
...
Executing <Task pending coro=<f() running at test_aio.py:12> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fe89344fcd8>()] created at /usr/lib/python3.6/asyncio/base_events.py:275> created at test_aio.py:13> took 0.100 seconds
...
```
What can be another way to schedule a lot of parallel tasks? |
|
Date |
User |
Action |
Args |
2018-03-21 15:48:58 | decaz | set | recipients:
+ decaz, asvetlov, yselivanov |
2018-03-21 15:48:58 | decaz | set | messageid: <1521647338.78.0.467229070634.issue33115@psf.upfronthosting.co.za> |
2018-03-21 15:48:58 | decaz | link | issue33115 messages |
2018-03-21 15:48:58 | decaz | create | |
|