Hi Demian, my intention is to demonstrate normal usage of Python’s HTTP client, whether or not its implementation misbehaves. I am trying to demonstrate a valid persistent server that happens to decide to close the connection after the first request but before reading a second request. Quoting the original post: “Servers may close a persistent connection after a request due to a timeout or other reason.” I am attaching a second demo script which includes short sleep() calls to emulate a period of time elapsing and the server timing out the connection, which is common for real-world servers.

The new script also avoids the EPIPE race by waiting until the server has definitely shut down the socket, and also demonstrates ECONNRESET. However this synchronization is artificial: in the real world the particular failure mode (BadStatusLine/EPIPE/ECONNRESET) may be uncertain.

If you are worried about detecting a misbehaving server that closes the connection before even responding to the first request, perhaps the HTTPConnection class could maintain a flag and handle the closed connection differently if it has not already seen a complete response.

If you are worried about detecting a misbehaving server that sends an empty status line without closing the connection, there will still be a newline code received. This is already handled separately by existing code: Lib/http/client.py:210 versus Lib/http/client.py:223.

I think there should be a separate exception, say called ConnectionClosed, for when the “status line” is an empty string (""), which is caused by reading the end of the stream. This is valid HTTP behaviour for the second and subsequent requests, so the HTTP library should understand it. BadStatusLine is documented for “status codes we don’t understand”. The new ConnectionClosed exception should probably be a subclass of BadStatusLine for backwards compatibility.

A further enhancement could be to wrap any ConnectionError during the request() stage, or first part of the getresponse() stage, in the same ConnectionClosed exception. Alternatively, the new ConnectionClosed exception could subclass both BadStatusLine and ConnectionError. Either way, code like the XML-RPC client could be simplified to:

try:
    return self.single_request(...)
except http.client.ConnectionClosed:
#except ConnectionError:  # Alternative
    #retry request once if cached connection has gone cold
    return self.single_request(...)