Message237151
Initially reported on the requests bug list at https://github.com/kennethreitz/requests/issues/2467
In cases when a remote web server sends a non-chunked response that does not have a content length, it is possible to get http.client to hang on a read. To hit it requires a specific set of circumstances:
- Python 3 (this bug is not seen on Python 2 in my testing)
- HTTPS (using plaintext HTTP resolves the problem)
- A socket timeout must be set (leaving it unset, i.e. leaving the socket in blocking mode, resolves this problem)
- The server must not send a content-length or set Transfer-Coding: chunked.
- The reads must not be an even divisor of the content's length.
- The timeout must be longer than the time the server is willing to keep the connection open (otherwise an exception is thrown).
The following code can be used as a sample to demonstrate the bug:
import http.client
import time
def test(connection_class=http.client.HTTPSConnection, timeout=10, read_size=7):
start = time.time()
c = connection_class('sleepy-reaches-6892.herokuapp.com')
c.connect()
if timeout is not None:
c.sock.settimeout(timeout)
c.request('GET', '/test')
r = c.getresponse()
while True:
data = r.read(read_size)
if not data:
break
print('Finished in {}'.format(time.time() - start))
Below are the results from several different runs:
test(): Finished in 4.8294830322265625
test(connection_class=http.client.HTTPConnection): Finished in 0.3060309886932373
test(timeout=None): Finished in 0.6070599555969238
test(read_size=2): Finished in 0.6600658893585205
As you can see, only this particular set of features causes the bug. Note that if you change the URL to one that does return a Content-Length (e.g. http2bin.org/get), this bug also goes away.
This is a really weird edge case bug. There's some extensive investigation over at the requests bug report, but I've not been able to convincingly point at the problem yet. |
|
Date |
User |
Action |
Args |
2015-03-03 16:56:13 | Lukasa | set | recipients:
+ Lukasa |
2015-03-03 16:56:13 | Lukasa | set | messageid: <1425401773.41.0.539386605796.issue23576@psf.upfronthosting.co.za> |
2015-03-03 16:56:13 | Lukasa | link | issue23576 messages |
2015-03-03 16:56:12 | Lukasa | create | |
|