This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Python's CGIHTTPServer does not handle Expect: 100-continue gracefully which results in some Post requests being handled slowly.
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: duplicate
Dependencies: Superseder:
Assigned To: Nosy List: Joseph.Warren, jcea, orsenthil, r.david.murray
Priority: normal Keywords:

Created on 2013-09-19 15:26 by Joseph.Warren, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
Wireshark_Captures.html Joseph.Warren, 2013-09-19 15:26 Wireshark Capture data showing what's sent over the wire in three cases
Messages (9)
msg198084 - (view) Author: Joseph Warren (Joseph.Warren) * Date: 2013-09-19 15:26
I will add as a disclaimer to this bug report that I am relatively new to both the http spec, and Python programming, so I may have completely misunderstood something.

When I make a post request with some data (using libCurl), It sends the headers first, including one header, "Expect: 100-continue" and then waits for around a second, or until receiving a "100 Continue" response, before sending the rest of the data. (I have attached WireShark capture data as html to show this, I've also attached a dump showing how Apache behaves given the same request). 

This means that when I make such a request to a CGI script, hosted using CGIHTTPServer, and the script tries to read in the data, (for instance, using cgi.FieldStorage() ). Then the CGI script takes around a second to parse the data, since it hasn't been sent yet.

I currently have a work-around for this, which is to override CGIHTTPRequestHandler, and have it send a "100 Continue" response before doing anything else. A dump showing this is attached, this makes the connection much faster. This is fine for my application, as all CGI requests will be made in the same way, However it is not a general solution. 

The W3C, when defining Expect Headers, state (taken from http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html);

A server that does not understand or is unable to comply with any of the expectation values in the Expect field of a request MUST respond with appropriate error status. The server MUST respond with a 417 (Expectation Failed) status if any of the expectations cannot be met or, if there are other problems with the request, some other 4xx status. 


I would like to offer to implement handling of the Expect header in BaseHTTPServer, however I would like some feedback before doing this, and to check that the current behaviour is not intended for some reason.

This change would make CGIHTTPServer much faster in my use case, and I suspect it would help other people.
msg198086 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-09-19 15:44
See also issue 1346874.  It seems that we do not currently handle 'continue' correctly in general.
msg198088 - (view) Author: Joseph Warren (Joseph.Warren) * Date: 2013-09-19 15:51
Issue1346874 seems similar, but is talking about a failure to handle 100 Continue responses sent by a server to the client, this issue is in server code, and is a failure of BaseHttpServer to send 100 Continue responses to a client which expects them.

Please tell me if I'm mistaken/not making sense.
msg198090 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-09-19 15:57
You are making perfect sense.  My point in referencing that issue was that you are not crazy, we do indeed not handle continue correctly ;)
msg198091 - (view) Author: Joseph Warren (Joseph.Warren) * Date: 2013-09-19 16:01
Cheers, would you suggest I submit a patch then?

Also what version of python should I do this against? I've been working with 2.7, but the issue still exists in 3.* and I can conveniently work against 3.2.3-7, and less conveniently work against other versions.
msg198092 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-09-19 16:10
I think we can fix this in maintenance releases without breaking anyone's code, so a patch against both 2.7 and 3.3 would be ideal.  A patch against 3.2 will probably apply cleanly to 3.3, so that should be fine as well.

Thanks for working on this, and could you please submit a contributor agreement at your earliest convenience?  It takes a few days for that to get processed.
msg198117 - (view) Author: Joseph Warren (Joseph.Warren) * Date: 2013-09-19 22:36
Have downloaded an up to date version of Python to develop with, and found that this issue had been fixed in it. This seems to have been done in changeset:   65028:7add45bcc9c6

changeset:   65028:7add45bcc9c6
user:        Senthil Kumaran <orsenthil@gmail.com>
date:        Thu Sep 30 06:09:18 2010 +0000
summary:     Issue1491 - BaseHTTPServer incorrectly implements response code 100

This change does not seem to appear in 2.7 , and could be converted to work with this, however failing this, this issue can probably be closed.

I appologise for not spotting this earlier.
msg198118 - (view) Author: Joseph Warren (Joseph.Warren) * Date: 2013-09-19 22:37
Sorry, this change does appear in 2.7.* I was looking at the wrong mercurial Tag
msg198125 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-09-20 01:21
No need to apologize for missing the issue.  I missed it too, and I was looking for it because I thought I remembered something...but only found the one I referenced and assumed that that was what I was remembering.
History
Date User Action Args
2022-04-11 14:57:51adminsetgithub: 63252
2013-09-30 23:44:03jceasetnosy: + jcea
2013-09-20 01:21:58r.david.murraysettype: enhancement -> behavior
messages: + msg198125
stage: resolved
2013-09-19 22:37:39Joseph.Warrensetstatus: open -> closed
resolution: duplicate
2013-09-19 22:37:20Joseph.Warrensetmessages: + msg198118
2013-09-19 22:36:12Joseph.Warrensetmessages: + msg198117
2013-09-19 16:10:36r.david.murraysetmessages: + msg198092
2013-09-19 16:01:31Joseph.Warrensetmessages: + msg198091
2013-09-19 15:57:17r.david.murraysetmessages: + msg198090
2013-09-19 15:51:10Joseph.Warrensetmessages: + msg198088
2013-09-19 15:44:09r.david.murraysetnosy: + r.david.murray
messages: + msg198086
2013-09-19 15:29:53pitrousetnosy: + orsenthil

type: performance -> enhancement
versions: - Python 2.6, Python 3.1, Python 2.7, Python 3.2, Python 3.3, Python 3.5
2013-09-19 15:26:56Joseph.Warrencreate