Author martin.panter
Recipients Rotkraut, demian.brecht, harobed, martin.panter, orsenthil, petri.lehtinen, piotr.dobrogost, pitrou, whitemice
Date 2015-02-15.05:58:26
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1423979907.38.0.188212360891.issue12319@psf.upfronthosting.co.za>
In-reply-to
Content
One interesting question is how to convey data to the chunked encoder. There are two sets of options in my mind, a pull interface:

* iterable: seems to be the popular way amoung commenters here
* file reader object: encoder calls into stream’s read() method

and a push interface:

* chunked encoder is a file writer object: user calls encoder’s write() and close() methods. This would suit APIs like saxutils.XMLGenerator and TextIOWrapper.
* chunked encoder has a “feed parser” interface, codecs.IncrementalEncoder interface, or something else.

The advantage of the push interface is that you could fairly easily feed data from an iterable or file reader into it simply by just doing shutil.copyfileobj() or equivalent. But to adapt the pull interface to a push interface would require “asyncio” support or a separate thread or something to invert the flow of control. So I think building the encoder with a push interface would be best. Rolf’s ChunkedHTTPConnectionMixin class appears to only support the pull interface (iterables and stream readers).

I would welcome support for chunked uploading in Python’s “http.client” module, especially with push or stream writer support. I don’t think overwriting _send_request should be necessary; just call putrequest(), putheader() etc manually, and then call send() for each chunk. Perhaps there is scope for sharing the code with the “http.server” module (for encoding chunked responses).
History
Date User Action Args
2015-02-15 05:58:27martin.pantersetrecipients: + martin.panter, orsenthil, pitrou, harobed, petri.lehtinen, piotr.dobrogost, demian.brecht, whitemice, Rotkraut
2015-02-15 05:58:27martin.pantersetmessageid: <1423979907.38.0.188212360891.issue12319@psf.upfronthosting.co.za>
2015-02-15 05:58:27martin.panterlinkissue12319 messages
2015-02-15 05:58:26martin.pantercreate