Author vstinner
Recipients YoSTEALTH, benjamin.peterson, giampaolo.rodola, martin.panter, njs, pitrou, stutzbach, vstinner, xgdomingo
Date 2019-10-11.12:42:55
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
Here a proof-of-concept of an asynchronous io module reusing the existing blocking io module: it implements AsyncBufferedReader.readline() using existing _pyio.BufferedReader.readline().

The approach seems to work, but only if bpo-13322 is fixed first: BufferedReader, TextIOWrapper & friends must return None if the underlying object ("raw" and "buffer" objects) return None.


My PoC uses 3 classes:

* AsyncFileIO: similar to io.FileIO but uses "async def"
* AsyncBufferedReader: similar to io.BufferedReader but uses "async def"
* FileIOSandwich: glue between asynchronous AsyncFileIO and blocking io.BufferedReader

At the first read, call returns None, but it stores the request read size (n).

If AsyncBufferedReader gets None, is calls FileIOSandwich._prepare_read() *asynchronously*/

Then is called again, and this time it no longer blocks, since data has been already read.


Since bpo-13322 is not fixed, my PoC uses _pyio since it's easier to fix. It needs the following fix for _pyio.BufferedReader.readline():

diff --git a/Lib/ b/Lib/
index c1bdac7913..e90742ec43 100644
--- a/Lib/
+++ b/Lib/
@@ -557,6 +557,8 @@ class IOBase(metaclass=abc.ABCMeta):
         res = bytearray()
         while size < 0 or len(res) < size:
             b =
+            if b is None and not res:
+                return None
             if not b:
             res += b



$ ./python 
data: b'import asyncio\n'

Internally, _prepare_read() reads 8 KiB.
Date User Action Args
2019-10-11 12:42:55vstinnersetrecipients: + vstinner, pitrou, giampaolo.rodola, benjamin.peterson, stutzbach, njs, martin.panter, YoSTEALTH, xgdomingo
2019-10-11 12:42:55vstinnersetmessageid: <>
2019-10-11 12:42:55vstinnerlinkissue32561 messages
2019-10-11 12:42:55vstinnercreate