classification
Title: http.server should correctly handle HTTP 1.1 responses without a content-length
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Tom Forbes, martin.panter
Priority: normal Keywords:

Created on 2017-07-29 14:15 by Tom Forbes, last changed 2017-07-31 23:19 by Tom Forbes.

Messages (3)
msg299471 - (view) Author: Tom Forbes (Tom Forbes) Date: 2017-07-29 14:15
The builtin http.server module does not support HTTP keep-alive when sending a response without a content-length. This causes any clients to hang waiting on more response data, while the server hangs waiting for the client to send another request. This is documented (https://docs.python.org/3/library/http.server.html#http.server.BaseHTTPRequestHandler.protocol_version), but it is confusing.

As far as I can tell the fix would be pretty simple: If no content-length header is set then close the connection regardless of the keep-alive header (Keep-alive is advisory and servers can close the connection at-will, regardless of what the client sends).

If a response contains an inaccurate content-length header there is nothing we can do, but if none is present the server (and clients) should not just hang.
msg299473 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2017-07-29 15:09
The trouble is you would also have to parse the Transfer-Encoding field, and have special logic for responses where Content-Length is not needed or irrelevant (certain combinations of method and status code). And even then you risk breaking rare or custom methods and status codes. All this seems complex and at the wrong layer. A server shouldn’t be parsing the header fields it just generated.

Perhaps there could be a new HTTP 1.1 mode (separate from protocol_version) that still closed the connection by default, but had a clearer API for keeping the connection open that the programmer can use in the right circumstances. But I had this thought before (see Issue 21224), and it didn’t seem beneficial.

What’s your use case? Why not just stick with HTTP 1.0, or update the server code to either close the connection or use chunked encoding?
msg299589 - (view) Author: Tom Forbes (Tom Forbes) Date: 2017-07-31 23:19
Django recently switched to HTTP 1.1 on their development server, and it works fine as long as the middleware is included that generates the content length. Using a 'bare' Django project with no middleware the server will just hang.

It sounds like the easiest fix is to disable keepalive support in the development server, which sounds like a reasonable fix (it doesn't have that much benefit locally I think?).

It was certainly surprising to find this issue, I know it is documented but that could certainly be improved. It's currently just a plain line of text in quite a text-heavy page, perhaps it should be put in a warning box?
History
Date User Action Args
2017-07-31 23:19:22Tom Forbessetmessages: + msg299589
2017-07-29 15:09:27martin.pantersetnosy: + martin.panter
messages: + msg299473
2017-07-29 14:15:08Tom Forbescreate