New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HTTPS reads can block when content length not available and timeout set. #67764
Comments
Initially reported on the requests bug list at https://github.com/kennethreitz/requests/issues/2467 In cases when a remote web server sends a non-chunked response that does not have a content length, it is possible to get http.client to hang on a read. To hit it requires a specific set of circumstances:
The following code can be used as a sample to demonstrate the bug: import http.client
import time
def test(connection_class=http.client.HTTPSConnection, timeout=10, read_size=7):
start = time.time()
c = connection_class('sleepy-reaches-6892.herokuapp.com')
c.connect()
if timeout is not None:
c.sock.settimeout(timeout)
c.request('GET', '/test')
r = c.getresponse()
while True:
data = r.read(read_size)
if not data:
break
print('Finished in {}'.format(time.time() - start)) Below are the results from several different runs: test(): Finished in 4.8294830322265625 As you can see, only this particular set of features causes the bug. Note that if you change the URL to one that does return a Content-Length (e.g. http2bin.org/get), this bug also goes away. This is a really weird edge case bug. There's some extensive investigation over at the requests bug report, but I've not been able to convincingly point at the problem yet. |
Reproducing seems a bit irregular. Note that the last bytestring (the empty bytestring) is what takes time to read. Also note that HTTPResponse is a buffered I/O object, so normally you don't need to read up to the empty string. You can stop when you got less bytes than what you asked for. |
Varying reproduceability may have to do with sleepy-reaches-6892.herokuapp.com resolving to different endpoints (that domain name has a stupidly small TTL, by the way). Anyway, for an unknown reason the following patch seems to fix the issue. |
This is now fixed in all branches. Thanks for the report! |
New changeset 371cf371a6a1 by Antoine Pitrou in branch '2.7': |
New changeset 01cf9ce75eda by Antoine Pitrou in branch '3.4': New changeset fc0201ccbcd4 by Antoine Pitrou in branch 'default': |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: