Author yselivanov
Recipients asvetlov, gvanrossum, pitrou, yselivanov
Date 2017-12-08.02:10:38
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1512699039.22.0.213398074469.issue32251@psf.upfronthosting.co.za>
In-reply-to
Content
A couple emails from async-sig for the context:

1. https://mail.python.org/pipermail/async-sig/2017-October/000392.html
2. https://mail.python.org/pipermail/async-sig/2017-December/000423.html

I propose to add another Protocol base class to asyncio: BufferedProtocol.  It will have 'get_buffer()' and 'buffer_updated(nbytes)' methods instead of 'data_received()':

    class asyncio.BufferedProtocol:

        def get_buffer(self) -> memoryview:
            pass

        def buffer_updated(self, nbytes: int):
            pass

When the protocol's transport is ready to receive data, it will call `protocol.get_buffer()`.  The latter must return an object that implements the buffer protocol.  The transport will request a writable buffer over the returned object and receive data *into* that buffer.

When the `sock.recv_into(buffer)` call is done, `protocol.buffer_updated(nbytes)` method will be called.  The number of bytes received into the buffer will be passed as a first argument.

I've implemented the proposed design in uvloop (branch 'get_buffer', [1]) and adjusted your benchmark [2] to use it.  Here are benchmark results from my machine (macOS):

vanilla asyncio: 120-135 Mb/s
uvloop: 320-330 Mb/s
uvloop/get_buffer: 600-650 Mb/s.


[1] https://github.com/MagicStack/uvloop/tree/get_buffer
[2] https://gist.github.com/1st1/1c606e5b83ef0e9c41faf21564d75ad7
History
Date User Action Args
2017-12-08 02:10:39yselivanovsetrecipients: + yselivanov, gvanrossum, pitrou, asvetlov
2017-12-08 02:10:39yselivanovsetmessageid: <1512699039.22.0.213398074469.issue32251@psf.upfronthosting.co.za>
2017-12-08 02:10:39yselivanovlinkissue32251 messages
2017-12-08 02:10:38yselivanovcreate