Author njs
Recipients asvetlov, gvanrossum, lukasz.langa, njs, xtreak, yselivanov
Date 2019-09-21.08:55:55
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1569056156.13.0.777903660175.issue38242@roundup.psfhosted.org>
In-reply-to
Content
That seems a bit extreme, and I don't think conflicts with Trio are the most important motivation here. (And they shouldn't be.) But, we should probably write down what the motivations actually are.

Yury might have a different perspective, but to me the challenge of the asyncio stream API is that it's trying to do three things at once:

1. Provide a generic interface for representing byte streams
2. Provide useful higher-level tools for working with byte streams, like readuntil and TLS
3. Adapt the protocol/transports world into the async/await world

These are all important things that should exist. The problem is that asyncio.Stream tries to do all three at once in a single class, which makes them tightly coupled.

*If* we're going to have a single class that does all those things, then I think the new asyncio.Stream code does that about as well as it can be done.

The alternative is to split up those responsibilities: solve (1) by defining some simple ABCs to represent a generic stream, that are optimized to be easy to implement for lots of different stream types, solve (2) by building higher-level tools that work against the ABC interface so they work on any stream implementation, and then once that's done it should be pretty easy to solve (3) by implementing the new ABC for protocol/transports.

By splitting things up this way, I think we can do a better job at solving all the problems with fewer compromises. For example, there are lots of third-party libraries that use asyncio and want to expose some kind of stream object, like HTTP libraries, SSH libraries, SOCKS libraries, etc., but the current design makes this difficult. Also, building streams on top of protocols/transports adds unnecessary runtime overhead.

A further bonus is that (1) and (2) aren't tied to any async library, so it's possible to borrow the work that Trio's been doing on them, and to potentially share this part of the code between Trio and asyncio.

So that all sounds pretty cool, but what does it have to do with shipping asyncio.Stream in 3.8? The concern is that right now asyncio has two APIs for working with byte streams (protocols/transports + StreamReader/StreamWriter); then 3.8 adds asyncio.Stream; and then the design I'm describing will make a *fourth*, which is a lot. I think we need something *like* asyncio.Stream in the mix, for compatibility and bridging between the two worlds, but right now it's not quite clear how that should look, and pushing it off to 3.9 would give us time to figure out the details for how these things can all fit together best.
History
Date User Action Args
2019-09-21 08:55:56njssetrecipients: + njs, gvanrossum, asvetlov, lukasz.langa, yselivanov, xtreak
2019-09-21 08:55:56njssetmessageid: <1569056156.13.0.777903660175.issue38242@roundup.psfhosted.org>
2019-09-21 08:55:56njslinkissue38242 messages
2019-09-21 08:55:55njscreate