classification
Title: httplib.HTTPResponse.read() not updating the length correctl
Type: Stage:
Components: Library (Lib) Versions: Python 2.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jhylton Nosy List: gvanrossum, irmen, jhylton, jjlee, jpe, misa
Priority: normal Keywords:

Created on 2004-07-09 17:37 by misa, last changed 2004-11-07 16:16 by jhylton.

Messages (6)
msg21462 - (view) Author: Mihai Ibanescu (misa) Date: 2004-07-09 17:37 
I believe there is a bug in HTTPResponse.read. This is
the last part:
                                                      
                         
                                                      
                         
                                                      
                         
        if self.length is not None:
            if amt > self.length:
                # clip the read to the "end of response"
                amt = self.length
            self.length -= amt
                                                      
                         
        # we do not use _safe_read() here because this
may be a .will_close
        # connection, and the user is reading more
bytes than will be provided
        # (for example, reading in 1k chunks)
        s = self.fp.read(amt)
                                                      
                         
        return s
                                                      
                         
                                                      
                         
self.length is the response's length as presented by
the Content-Length header.
                                                      
                         
If I am not mistaken, self.fp.read(amt) (for the case
where amt is not None) returns _up to_, not _exactly_
amt bytes.
                                                      
                         
So, if amt is originally 1024 bytes and self.fp.read
returns only 520 bytes (because, let's say, you are
reading from an SSL socket), then self.length is
decremented by 1024 and not by 520. So you end up
reading less than self.length
                                                      
                         
I am seeing this when plugging an SSL socket/file from
pyOpenSSL.
msg21463 - (view) Author: Guido van Rossum (gvanrossum) * Date: 2004-07-10 03:31 
Logged In: YES 
user_id=6380

Python's regular files always return the requested amount
except at EOF. But many file-like objects don't guarantee
this, so I agree that it's better to only adjust the
position by the number of bytes actually read.
msg21464 - (view) Author: John J Lee (jjlee) Date: 2004-07-10 22:30 
Logged In: YES 
user_id=261020

Patch 988642 addresses this.
msg21465 - (view) Author: John Ehresman (jpe) Date: 2004-07-10 22:42 
Logged In: YES 
user_id=22785

Submitted patch #988642 to fix this
msg21466 - (view) Author: Irmen de Jong (irmen) Date: 2004-11-07 15:30 
Logged In: YES 
user_id=129426

slightly improved patch at 1061941
msg21467 - (view) Author: Jeremy Hylton (jhylton) Date: 2004-11-07 16:16 
Logged In: YES 
user_id=31392

Fixed in httplib.py 1.94 by patch 1061941.
Bug fix candidate.
History
Date User Action Args
2004-07-09 17:37:06misacreate