Title: cgitb sends a bogus HTTP header if the app crashes before finishing headers
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.5, Python 3.2, Python 3.3, Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: meatballhat, orsenthil, p0lar_bear, stutzbach, ysj.ray, Артур Клесун
Priority: normal Keywords: easy

Created on 2010-05-13 14:12 by stutzbach, last changed 2017-05-20 10:40 by Артур Клесун.

Messages (5)
msg105633 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2010-05-13 14:12
If the CGI script crashes before finishing the headers, cgitb will emit invalid HTTP headers before showing the error message.  Below are HTTP headers I received, captured with a packet sniffer.  Note the "<--: spam".

HTTP/1.1 200 OK
Date: Thu, 13 May 2010 14:00:42 GMT
Server: Apache/2.2.9
<!--: spam
Vary: Accept-Encoding
Cache-Control: max-age=0
Expires: Thu, 13 May 2010 14:00:42 GMT
Set-Cookie: ref=; path=/; HttpOnly
Transfer-Encoding: chunked
Content-Type: text/html

That string it emitted by cgitb.reset(), which is trying to reset the browser to a sane state so the error message will be shown.  The problem can be easily fixed by having cgitb.reset() emit two CRLF pairs first, to ensure that we're done with the headers and emitting content:

-    return '''<!--: spam
+    return '''\r\n\r\n<!--: spam
msg105690 - (view) Author: ysj.ray (ysj.ray) Date: 2010-05-14 07:57
Yes, I saw the "<!--: spam" string in headers, but it seems that this string doesn't make problems. The displaying page is correct.

But after I apply the changes you mentioned:
-    return '''<!--: spam
+    return '''\r\n\r\n<!--: spam

I got text/plain output, and the response headers are like this:

    Date	Fri, 14 May 2010 07:30:03 GMT
    Server	Apache/2.2.15 (Unix)
    Keep-Alive	timeout=5, max=100
    Connection	Keep-Alive
    Transfer-Encoding	chunked
    Content-Type	text/plain

And the content is like this:

<!--: spam
Content-Type: text/html

<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> -->
<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> --> -->


So the hole page is not displayed correctly!

Is there any problem with me?
msg105705 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2010-05-14 13:06
It displays correctly in some browsers, yes, but not everything that speaks HTTP is a browser.  For example, the invalid header makes C#'s WebRequest throw an exception.

I hadn't noticed the 'Content-Type' on the next line of the string output by reset().  That does make things more complicated.

We could put the "Content-Type: text/html" first, but the downside is that it will be output as visible content if a script crashes after the headers have been emitted.

I'm not sure if that's better or worse than emitting an invalid header.
msg193189 - (view) Author: p0lar_bear (p0lar_bear) Date: 2013-07-16 20:44
I get similar results if my CGI script sends a Content-Type header of anything besides "text/html", e.g. print('Content-Type: text/json').
msg294027 - (view) Author: Артур Клесун (Артур Клесун) Date: 2017-05-20 10:40
Apache started strict check of headers ch
aracters to be valid recently. That causes it fail on "<--: spam".

[Sat May 20 13:09:23.056673 2017] [http:error] [pid 26379] [client 12.34.567.41:60988] AH02429: Response header name '<!--' contains invalid characters, aborting request, referer:

The workaround is to put:
HttpProtocolOptions Unsafe
line into your apache .conf
Date User Action Args
2017-05-20 10:40:20Артур Клесунsetnosy: + Артур Клесун

messages: + msg294027
versions: + Python 3.5
2013-07-16 20:51:52p0lar_bearsetversions: + Python 3.3
2013-07-16 20:44:40p0lar_bearsetnosy: + p0lar_bear
messages: + msg193189
2011-03-18 02:11:43orsenthilsetnosy: + orsenthil
2010-05-14 13:06:35stutzbachsetmessages: + msg105705
2010-05-14 07:57:48ysj.raysetnosy: + ysj.ray
messages: + msg105690
2010-05-14 01:44:28meatballhatsetnosy: + meatballhat
2010-05-13 14:12:58stutzbachsetkeywords: + easy
2010-05-13 14:12:50stutzbachcreate