Author giampaolo.rodola
Recipients giampaolo.rodola, neologix, vstinner, yselivanov
Date 2017-04-08.18:31:39
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1491676300.7.0.951966566111.issue30014@psf.upfronthosting.co.za>
In-reply-to
Content
For the sake of experiment I'm attaching a toy echo server which uses modify() to switch between EVENT_READ and EVENT_WRITE. Without patch I get 35000 req/sec, with patch around 39000 req/sec (11.4% faster).
To be entirely honest a smarter echo server would switch between EVENT_READ / EVENT_WRITE only in case of BlockingIOError on send() or recv(), but unfortunately it's a condition which (for send()) does not occur often by testing on localhost (for recv() it naturally occurs +27000 times out of 39000). On the other hand, by experimenting with a real network it occurs quite soon:

import socket
sock = socket.socket()
sock.connect(('google.com', 80))
sock.setblocking(False)
print(sock.send(b'x' * 50000))
print(sock.send(b'x' * 50000))  # raise BlockingIOError

So basically my benchmark is emulating a worst case scenario in which send() always blocks on the first call and succeed on the next one, in order to mimic recv() which blocks half of the times.
The whole point is that the speedup entirely depends on how many times send() or recv() will block in a real world app connected to the internet (because that's when modify() is supposed to be used) but that's hard to determine, especially under heavy server loads which is hard to emulate. This also involves the SSL handshake when it fails with WANT_READ / WANT_WRITE, which is another circumstance where modify() is going to be used and for which I have no benchmarks.
I personally don't mind about selectors module per-se, but given that it's the core of important libs such as asyncio and curio I would like to hear a better rationale than "let's reject this 1.5x speedup because DRY does not apply in this case".
Also please consider that Tornado, Twisted and gevent use native modify() method.
History
Date User Action Args
2017-04-08 18:31:40giampaolo.rodolasetrecipients: + giampaolo.rodola, vstinner, neologix, yselivanov
2017-04-08 18:31:40giampaolo.rodolasetmessageid: <1491676300.7.0.951966566111.issue30014@psf.upfronthosting.co.za>
2017-04-08 18:31:40giampaolo.rodolalinkissue30014 messages
2017-04-08 18:31:40giampaolo.rodolacreate