Issue24486
Created on 2015-06-22 13:15 by Julien.Palard, last changed 2015-06-24 12:55 by martin.panter. This issue is now closed.
Messages (6) | |||
---|---|---|---|
msg245626 - (view) | Author: Julien Palard (Julien.Palard) | Date: 2015-06-22 13:15 | |
Requesting HTTP using `requests`, which uses `http.client` which use `socket`, sometimes, my program get stuck like this: ``` File "/usr/lib/python3.2/socket.py", line 287 in readinto File "/usr/lib/python3.2/http/client.py", line 308 in _read_status File "/usr/lib/python3.2/http/client.py", line 346 in begin File "/usr/lib/python3.2/http/client.py", line 1052 in getresponse ... in python requests ... ``` The line is the first of _read_status:: ``` line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") ``` At a lower level, the program is stuck in a `recvfrom()` syscall. So obviously, an error at the network level caused the server to not reply, but the client recvfrom() the network in a blocking socket without checking first if data is available, so it get stuck indefinitely :-( |
|||
msg245632 - (view) | Author: Martin Panter (martin.panter) * ![]() |
Date: 2015-06-22 14:39 | |
You did not mention if the HTTP connection has a timeout set. If no timeout is set, it will block until the server sends or closes the connection. The timeout sets how long the socket spends “checking if data is available” before giving up. |
|||
msg245669 - (view) | Author: Julien Palard (Julien.Palard) | Date: 2015-06-23 07:43 | |
I only have a `socket.setdefaulttimeout(10)` just after my imports... |
|||
msg245686 - (view) | Author: Martin Panter (martin.panter) * ![]() |
Date: 2015-06-23 12:49 | |
The closest I have is Python 3.3, but this times out properly for me: >>> from http.client import HTTPConnection >>> import socket >>> socket.setdefaulttimeout(10) >>> h = HTTPConnection("192.168.1.84") >>> h.request("GET", "/") >>> h.getresponse() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Python33\lib\http\client.py", line 1147, in getresponse response.begin() File "C:\Python33\lib\http\client.py", line 358, in begin version, status, reason = self._read_status() File "C:\Python33\lib\http\client.py", line 320, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "C:\Python33\lib\socket.py", line 297, in readinto return self._sock.recv_into(b) socket.timeout: timed out I suggest you either try to come up with a recipe to reproduce this with plain http.client, or look into whether the Requests package supports using the default socket timeout setting. |
|||
msg245735 - (view) | Author: Julien Palard (Julien.Palard) | Date: 2015-06-24 12:46 | |
OK, so, requests have a `timeout` and take it into account, and it solves my problem. Yet I don't understand one little thing: With both requests `timeout` parameter set or unset, the exact same http.client.py:_read_status call the same socket.readinto. With a `request` `timeout`, the socket.readinto uses the recvfrom syscall, but with a `request` `timeout`, readinto uses a `poll` syscall with the given timeout. The root of the problem is that urllib3 ignores the `socket` `defaulttimeout`, I opened a ticket on this: https://github.com/shazow/urllib3/issues/655#issuecomment-114835279 So this ticket can be considered closed. |
|||
msg245736 - (view) | Author: Martin Panter (martin.panter) * ![]() |
Date: 2015-06-24 12:55 | |
I assume you meant _without_ an explicit timeout setting in Requests, you see a recvfrom() system call, and _with_ an explicit timeout you see poll(). I guess that would be because Requests is using a blocking socket in the first case, and in the second case the timeout is implemented by waiting for data to be available before reading it. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2015-06-24 12:55:58 | martin.panter | set | resolution: third party messages: + msg245736 stage: resolved |
2015-06-24 12:46:28 | Julien.Palard | set | status: open -> closed |
2015-06-24 12:46:20 | Julien.Palard | set | messages: + msg245735 |
2015-06-23 12:49:50 | martin.panter | set | messages: + msg245686 |
2015-06-23 07:43:54 | Julien.Palard | set | messages: + msg245669 |
2015-06-22 14:39:29 | martin.panter | set | type: behavior messages: + msg245632 nosy: + martin.panter |
2015-06-22 13:15:11 | Julien.Palard | create |