New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide access to buffer of asyncio.StreamReader #76233
Comments
While asyncio.StreamReader.readuntil is an improvement on only having readline, it is still quite limited e.g. you cannot have multiple possible terminators. The real problem is that it's not possible to roll your own without accessing _underscore fields (other than by reading one byte at a time, which I'm guessing would be bad for performance). I'm not sure exactly what a public API to assist would look like, but I think the following would be a good start:
With that available, I think readuntil or more complex variants of it could be implemented externally using only the public interface (consumption of data from the buffer would be via readexactly rather than by messing with the buffer array directly). |
If the problm is in readuntil() functionality -- let's discuss th function improvement (in separate issue). Exposing streams internals is antipattern and very bad idea. Yury, what is your opinion? |
I'd be more comfortable with the idea of exposing the buffer when we have BufferedProtocol. Let's wait on this one. |
So we have BufferedProtocol in 3.7; now we need to re-implement asyncio streams on top of it. But even after doing that I'm not that sure we want to expose the low-level buffer. |
Exposing internal buffer means committing on a new API contract forever. I feel a need for reacher read*() API but pretty sure that making internal buffer public is a bad idea. With BufferedProtocol it could be even worse: SLAB allocators can spit a buffer into several separate chunks.
|
A sequence of possible terminators would cover my immediate use case and certainly be an improvement. To facilitate more general use cases without exposing implementation details, would it be practical and maintainable to have a "putback" method that prepends data to the buffer? It might not be fast in all cases (e.g. it might have to make a copy of what's still in the buffer), but possibly BufferedReader could detect the common case (putting back a suffix of what's just been read) and adjust its offsets into its internal buffer (although I'm not at all familiar with BufferedReader, so feel free to tell me I'm talking nonsense). |
Closing.
|
Ok, I'll open a separate issue to allow a tuple of possible separators. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: