classification
Title: Add StreamReader.readinto()
Type: enhancement Stage: patch review
Components: asyncio Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: aeros, asvetlov, njs, tontinton, yselivanov
Priority: normal Keywords: patch

Created on 2020-07-15 16:16 by tontinton, last changed 2020-07-16 21:34 by yselivanov.

Pull Requests
URL Status Linked Edit
PR 21491 open tontinton, 2020-07-15 17:41
Messages (10)
msg373702 - (view) Author: Tony (tontinton) * Date: 2020-07-15 16:16
Add a StreamReader.readinto(buf) function.

Exactly like StreamReader.read() with *n* being equal to the length of buf.

Instead of allocating a new buffer, copy the read buffer into buf.
msg373714 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2020-07-15 19:36
We don't want to extend StreamReader with new APIs as ultimately we plan to deprecate it. A new streams API is needed, perhaps inspired by Trio. Sadly, I'm -1 on this one.
msg373719 - (view) Author: Tony (tontinton) * Date: 2020-07-15 20:01
ok.

Im interested in learning about the new api.
Is it documented somewhere?
msg373720 - (view) Author: Tony (tontinton) * Date: 2020-07-15 20:02
Ah it's trio...
msg373732 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2020-07-15 23:25
> Im interested in learning about the new api.

There are two problems with the current API:

1. Reader and writer are separate objects, while they should be one.
2. Writer.write is not a coroutine, although it should be one.

There are other minor nits, but that's the crux of the problem. So we need a new streams design + a new set of APIs to work with it (and streams are in many places, like in the subprocess APIs).

Trio was going to stabilize their own streaming API and we thought it would be great if our new API was compatible with it (not sure if they did stabilize it or not).

I was going to lead the project myself (and still am) but dropped the ball and we missed 3.9 to do this. If you want to start working on this I'd be glad to assist with discussions & reviews.
msg373750 - (view) Author: Tony (tontinton) * Date: 2020-07-16 08:00
Ok actually that sounds really important, I am interested.

But to begin doing something like this I need to know what's the general design.

Is it simply combining stream reader and stream writer into a single object and changing the write() function to always wait the write (thus deprecating drain) and that's it?

If there is not more to it I can probably do this pretty quickly, I mean it seems easy on the surface.

If there is more to it then I would like a more thorough explanation. Maybe we should chat about this.
msg373765 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2020-07-16 19:49
> Is it simply combining stream reader and stream writer into a single object and changing the write() function to always wait the write (thus deprecating drain) and that's it?

Pretty much. We might also rename a few APIs here and there, like "close()" should become an "async def aclose()"

We also will likely want to define a few ABCs.

Which brings me to the most important point: what we need it not coding it (yet), but rather drafting the actual proposal and posting it to https://discuss.python.org/c/async-sig/20.  Once a formal proposal is there we can proceed with the implementation.

It's a bit of a process to follow, but we have to do it this way.
msg373770 - (view) Author: Tony (tontinton) * Date: 2020-07-16 21:00
> Which brings me to the most important point: what we need it not coding it (yet), but rather drafting the actual proposal and posting it to https://discuss.python.org/c/async-sig/20.  Once a formal proposal is there we can proceed with the implementation.

Posted: https://discuss.python.org/t/discussion-on-a-new-api-for-asyncio/4725

By the way I know it's unrelated but I want a CR on https://github.com/python/cpython/pull/21446 I think it's also very important.
msg373772 - (view) Author: Tony (tontinton) * Date: 2020-07-16 21:27
By the way if we will eventually combine StreamReader and StreamWriter won't this function (readinto) be useful then?

Maybe we should consider adding it right now.

Tell me your thoughts on this.
msg373773 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2020-07-16 21:34
> By the way if we will eventually combine StreamReader and StreamWriter won't this function (readinto) be useful then?

Yes.  But StreamReader and StreamWriter will stay for the foreseeable future for backwards compatibility pretty much frozen in time.

So I'd like to start treating them as legacy APIs now.
History
Date User Action Args
2020-07-16 21:34:43yselivanovsetmessages: + msg373773
2020-07-16 21:27:19tontintonsetmessages: + msg373772
2020-07-16 21:00:59tontintonsetmessages: + msg373770
2020-07-16 19:49:12yselivanovsetnosy: + njs, aeros
messages: + msg373765
2020-07-16 08:00:32tontintonsetmessages: + msg373750
2020-07-15 23:25:03yselivanovsetmessages: + msg373732
2020-07-15 20:02:17tontintonsetmessages: + msg373720
2020-07-15 20:01:29tontintonsetmessages: + msg373719
2020-07-15 19:36:09yselivanovsetmessages: + msg373714
2020-07-15 17:41:17tontintonsetkeywords: + patch
stage: patch review
pull_requests: + pull_request20634
2020-07-15 16:28:43xtreaksetnosy: + yselivanov, asvetlov

type: enhancement
components: + asyncio
versions: + Python 3.10
2020-07-15 16:16:43tontintoncreate