Author malinoff
Recipients asvetlov, malinoff, wumpus, yselivanov
Date 2018-05-26.04:32:19
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1527309140.16.0.682650639539.issue31096@psf.upfronthosting.co.za>
In-reply-to
Content
I've changed the implementation significantly since August 2017, futures are not involved anymore so please ignore that part. However, such drain behavior is still present - but I don't think anymore that yielding to the event loop is an easy fix.

I've tried to do so in my lib, and it showed significant slowdowns (around 4-5k publishes per second). It's not acceptable. I also found this message from Guido https://github.com/python/asyncio/issues/263#issuecomment-142702725.

What really helped is a counter that tracks send calls without waiting for replies, and a user-provided limit; when the counter reaches the limit, an explicit yield (via await asyncio.sleep(0)) is performed. This helped to achieve around 15-16k publishes per second (3-4 times faster. Here's the code:
https://github.com/malinoff/amqproto/blob/6568204b539ecf820af2da11bddcca9ce7323ac5/amqproto/adapters/asyncio_adapter.py#L53-L71

Now I'm thinking that such behavior should only be documented - so library authors can deal with it before they face this in production. But if you have other thoughts, I'd be glad to hear.
History
Date User Action Args
2018-05-26 04:32:20malinoffsetrecipients: + malinoff, asvetlov, yselivanov, wumpus
2018-05-26 04:32:20malinoffsetmessageid: <1527309140.16.0.682650639539.issue31096@psf.upfronthosting.co.za>
2018-05-26 04:32:20malinofflinkissue31096 messages
2018-05-26 04:32:19malinoffcreate