Message317726
I've changed the implementation significantly since August 2017, futures are not involved anymore so please ignore that part. However, such drain behavior is still present - but I don't think anymore that yielding to the event loop is an easy fix.
I've tried to do so in my lib, and it showed significant slowdowns (around 4-5k publishes per second). It's not acceptable. I also found this message from Guido https://github.com/python/asyncio/issues/263#issuecomment-142702725.
What really helped is a counter that tracks send calls without waiting for replies, and a user-provided limit; when the counter reaches the limit, an explicit yield (via await asyncio.sleep(0)) is performed. This helped to achieve around 15-16k publishes per second (3-4 times faster. Here's the code:
https://github.com/malinoff/amqproto/blob/6568204b539ecf820af2da11bddcca9ce7323ac5/amqproto/adapters/asyncio_adapter.py#L53-L71
Now I'm thinking that such behavior should only be documented - so library authors can deal with it before they face this in production. But if you have other thoughts, I'd be glad to hear. |
|
Date |
User |
Action |
Args |
2018-05-26 04:32:20 | malinoff | set | recipients:
+ malinoff, asvetlov, yselivanov, wumpus |
2018-05-26 04:32:20 | malinoff | set | messageid: <1527309140.16.0.682650639539.issue31096@psf.upfronthosting.co.za> |
2018-05-26 04:32:20 | malinoff | link | issue31096 messages |
2018-05-26 04:32:19 | malinoff | create | |
|