Title: BaseHTTPRequestHandler, update the protocol version to http 1.1 by default?
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.5
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: martin.panter, matrixise, orsenthil, vstinner
Priority: normal Keywords:

Created on 2014-04-14 20:43 by matrixise, last changed 2015-11-26 02:00 by martin.panter.

Messages (16)
msg216206 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2014-04-14 20:43

With this issue, I would like to ask you to use the last version of the HTTP protocol in the BaseHTTPRequestHandler for Python 3.5.
Because this one uses the version 1.0 of the protocol for the backward-compatibility.

When we develop an web app with Flask/Werkzeug or an other tool, the default version of 
the protocol is "HTTP/1.0". If we use Gunicorn, the protocol version is HTTP/1.1 and not 1.0, but this one can support the old version.

So, I propose to change the default version to the HTTP/1.1. It seems that the code of the server can handle this version without any problem.

HTTP 1.0 - - 1996
HTTP 1.1 - - 1999

In 2014, I think we can move to HTTP 1.1 by default.


msg216214 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2014-04-14 21:02
It may not just the be the version, but the capabilities. We have ensure that capabilities are met/added before updated the version. Thanks for filing the issue.
msg216219 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2014-04-14 21:08
gunicorn has an implementation of the HTTP/1.1 protocol, we can ask to the author of this project if we can use its code and reuse it in the standard library.
msg216767 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2014-04-18 01:00
Looking at Issue 430706 and revision 27f36f4bf525, there is concious support for HTTP 1.1 persistent connections. Apparently the 1.0 default is just for backwards compatibility.
msg217584 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2014-04-30 09:15
In this case, I suggest than by default, we use the version 1.1 of HTTP 
and not 1.0.

We are in 2014, I suppose that all the old http clients use the version 
1.1 of the protocol,
otherwise, the user can specify the version 1.0.
msg229246 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2014-10-13 14:38
msg254159 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2015-11-06 01:23

patience is one key to success ;-)
msg254163 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-06 02:30
What are the advantages of changing the default? Just that the user no longer has to set it manually?

What do you think of the problem mentioned in the documentation? If an existing HTTP 1.0 server that works fine in Python 3.5 were to suddenly have protocol_version="HTTP/1.1" forced by default, it sounds like all its responses may stop working (hang from the client’s POV) if they don’t explicitly close the connection.

One option could be to wrap the “wfile” stream in a chunk encoder, and insert Transfer-Encoding: chunked. I haven’t thought through this much, and it may not work very well because Python’s HTTP server is so low-level. This would basically be a HTTP 1.1 to HTTP 1.0 proxy.

Similarly, I understand HTTP 1.1 requires chunked encoding support for requests, but there is no support for that in the Python implementation. Existing servers accepting e.g. POST uploads would expect a Content-Length header, which would be hard to fake with a compatibility proxy (may need to buffer it all to count the bytes).

Another way might be a deprecation cycle, to force people using 3.6 to set protocol_version.
msg254164 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-06 02:39
Actually RFC 7230 says “A server may reject a request that contains a message body but not a Content-Length by responding with 411 (Length Required)”, so maybe it is only clients that have to support chunked decoding. So I take back my paragraph about POST requests.
msg254183 - (view) Author: Stéphane Wirtel (matrixise) * Date: 2015-11-06 11:38
After your remarks, maybe we can close this issue. It's not just a simple modification of a string.

Do you know if we want to support the HTTP 1.1 and 2.0 in the future, directly in CPython and not via an external library (gunicorn, ...)

What do you think ? we close this issue and open an other with "Support of HTTP 1.1 and 2.0" ?
msg254250 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-07 01:49
There is already Issue 23794 discussing HTTP 2. IMO it would be interesting to implement, but I’m not sure how appropriate it would be in Python’s standard library at this stage. But working on it could drive other improvements in the existing HTTP stuff, as Demian hinted.

What do you mean by supporting HTTP 1.1 in the future? The basic support is already there; you just have to manually enable it. Maybe there is scope for making 1.1 easier to use, perhaps with a BaseHTTP11RequestHandler (shorter: Http11Handler) class or something.
msg255343 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-11-25 13:27
For HTTP 2, I consider that the protocol is too young. I prefer to move fast on a library hosted on PyPI, rather than puting something is the stable and "frozen" stdlib.
msg255345 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-11-25 13:29
> Similarly, I understand HTTP 1.1 requires chunked encoding support for requests, but there is no support for that in the Python implementation.

Hum, I don't understand exactly this issue. I understood that Python only has a "partial" implementation of the HTTP 1.1 protocol for the server side, and maybe even for the client side.

Because of that, I don't think that it's good idea to switch to HTTP 1.1 by default. We have to implement missing features server and client side before.
msg255389 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-25 21:57
Victor, that chunked support for server requests is optional, and not actually required (my quote was wrong; see my message after the one you quoted).

The client already does HTTP 1.1 by default:
>>> http.client.HTTPConnection._http_vsn_str

It is already possible to make a working HTTP 1.1 server by setting protocol_version. What things do you think would make the HTTP 1.1 server implementation more complete?
msg255391 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-11-25 23:08
Sorry, I read the issue very quickly. I don't understand why the
default is not changed.
msg255398 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-11-26 02:00
Changing it could break existing code written for the HTTP 1.0 behaviour. The main effect mentioned at <> is that in HTTP 1.0 mode, a server may send a response without Content-Length, because its connection will be automatically shut down when it is done. In HTTP 1.1 mode, the server code has to manually set close_connection = True (or indicate the end of the response some other way).

Looking at the source code, changing protocol_version to "HTTP/1.1":

* Allows close_connection to be False by default (causing the problem above)
* Enables Expect: 100-continue handling (no problem)
* Changes the version actually used in the HTTP protocol (probably acceptable if everything else keeps working with a HTTP 1.1 client)
* Affects the SERVER_PROTOCOL variable for CGI scripts (unsure of the consequences)

Perhaps one way forward would be to use HTTP 1.1 by default, but still set close_connection = True. But then someone may come along and say Python should support persistent connections by default! It does not seem worth enabling HTTP 1.1 with non-persistent connections as the default.
Date User Action Args
2015-11-26 02:00:27martin.pantersetmessages: + msg255398
2015-11-25 23:08:33vstinnersetmessages: + msg255391
2015-11-25 21:57:53martin.pantersetmessages: + msg255389
2015-11-25 13:29:41vstinnersetmessages: + msg255345
2015-11-25 13:27:17vstinnersetnosy: + vstinner
messages: + msg255343
2015-11-07 01:49:39martin.pantersetmessages: + msg254250
2015-11-06 11:38:33matrixisesetmessages: + msg254183
2015-11-06 02:39:08martin.pantersetmessages: + msg254164
2015-11-06 02:30:37martin.pantersettype: enhancement
messages: + msg254163
2015-11-06 01:23:36matrixisesetmessages: + msg254159
2014-10-13 14:38:45matrixisesetmessages: + msg229246
2014-04-30 09:15:15matrixisesetmessages: + msg217584
2014-04-18 01:00:30martin.pantersetnosy: + martin.panter
messages: + msg216767
2014-04-14 21:08:15matrixisesetmessages: + msg216219
2014-04-14 21:02:52orsenthilsetnosy: + orsenthil
messages: + msg216214
2014-04-14 20:43:52matrixisecreate