classification
Title: urllib.request and urllib.response cannot handle HTTP1.1 chunked encoding
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.0
process
Status: closed Resolution: duplicate
Dependencies: Superseder: urlopen returns extra, spurious bytes
View: 4631
Assigned To: Nosy List: chrisleow, jhylton (2)
Priority: Keywords

Created on 2008-09-02 20:59 by chrisleow, last changed 2008-12-15 03:59 by jhylton.

Messages (2)
msg72366 - (view) Author: Chris Leow (chrisleow) Date: 2008-09-02 20:59 
Hi, fairly new to Python, so not sure if this is something you want as a
behaviour or not:

urllib.response object when fetching an HTTP1.1 page does not
transparently handle "Transfer-Encoding": "chunked", and I think it should.

You can view source code for addinfourl, AbstractHTTPHandler and
HTTPHandler to verify this (sorry, I don't have line-numbers, I'm typing
this at home).

I would suggest extending addinfourl to "addinfourlchunked", for
example, to allow substitutes for fp.read(), readlines() and readline()
to be specified during the construction of addinfourl.

This threw me initially, and seems like quite a glareing omission for
newbies.

Cheers,

Chris
msg77845 - (view) Author: Jeremy Hylton (jhylton) Date: 2008-12-15 03:59 
I'm sorry that I didn't notice this bug report in September!  The
chunked support does exist in the http package, but it doesn't work with
urllib.  Tracking in 4631.
History
Date User Action Args
2008-12-15 03:59:44jhyltonsetstatus: open -> closed
resolution: duplicate
superseder: urlopen returns extra, spurious bytes
messages: + msg77845
nosy: + jhylton
2008-09-02 20:59:47chrisleowcreate