This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: cgitb sends a bogus HTTP header if the app crashes before finishing headers
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Ryan Tu, coyot, cthart, igs, meatballhat, orsenthil, p0lar_bear, stutzbach, ysj.ray, Артур Клесун
Priority: normal Keywords: easy

Created on 2010-05-13 14:12 by stutzbach, last changed 2022-04-11 14:57 by admin.

Messages (9)
msg105633 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2010-05-13 14:12
If the CGI script crashes before finishing the headers, cgitb will emit invalid HTTP headers before showing the error message.  Below are HTTP headers I received, captured with a packet sniffer.  Note the "<--: spam".

HTTP/1.1 200 OK
Date: Thu, 13 May 2010 14:00:42 GMT
Server: Apache/2.2.9
<!--: spam
Vary: Accept-Encoding
Cache-Control: max-age=0
Expires: Thu, 13 May 2010 14:00:42 GMT
Set-Cookie: ref=; path=/; HttpOnly
Transfer-Encoding: chunked
Content-Type: text/html

That string it emitted by cgitb.reset(), which is trying to reset the browser to a sane state so the error message will be shown.  The problem can be easily fixed by having cgitb.reset() emit two CRLF pairs first, to ensure that we're done with the headers and emitting content:

-    return '''<!--: spam
+    return '''\r\n\r\n<!--: spam
msg105690 - (view) Author: ysj.ray (ysj.ray) Date: 2010-05-14 07:57
Yes, I saw the "<!--: spam" string in headers, but it seems that this string doesn't make problems. The displaying page is correct.


But after I apply the changes you mentioned:
-    return '''<!--: spam
+    return '''\r\n\r\n<!--: spam


I got text/plain output, and the response headers are like this:

    Date	Fri, 14 May 2010 07:30:03 GMT
    Server	Apache/2.2.15 (Unix)
    Keep-Alive	timeout=5, max=100
    Connection	Keep-Alive
    Transfer-Encoding	chunked
    Content-Type	text/plain



And the content is like this:

<!--: spam
Content-Type: text/html

<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> -->
<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> --> -->

......



So the hole page is not displayed correctly!

Is there any problem with me?
msg105705 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2010-05-14 13:06
It displays correctly in some browsers, yes, but not everything that speaks HTTP is a browser.  For example, the invalid header makes C#'s WebRequest throw an exception.

I hadn't noticed the 'Content-Type' on the next line of the string output by reset().  That does make things more complicated.

We could put the "Content-Type: text/html" first, but the downside is that it will be output as visible content if a script crashes after the headers have been emitted.

I'm not sure if that's better or worse than emitting an invalid header.
msg193189 - (view) Author: p0lar_bear (p0lar_bear) Date: 2013-07-16 20:44
I get similar results if my CGI script sends a Content-Type header of anything besides "text/html", e.g. print('Content-Type: text/json').
msg294027 - (view) Author: Артур Клесун (Артур Клесун) Date: 2017-05-20 10:40
Apache started strict check of headers ch
aracters to be valid recently. That causes it fail on "<--: spam".

```
[Sat May 20 13:09:23.056673 2017] [http:error] [pid 26379] [client 12.34.567.41:60988] AH02429: Response header name '<!--' contains invalid characters, aborting request, referer: http://example.com/
```

http://apache-http-server.18135.x6.nabble.com/Bug-60863-New-Apache-proxy-2-4-25-can-disable-header-check-Set-Cookie-td5036120.html

The workaround is to put:
HttpProtocolOptions Unsafe
line into your apache .conf
msg364186 - (view) Author: Ryan Tu (Ryan Tu) Date: 2020-03-14 15:51
#Maybe not a good solution
I do not know the should we delete the code in cgitb.py or adjust the configration of apache httpd. My solution is deleting some code as follows:
```
        return '''
<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> -->
<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> --> -->
</font> </font> </font> </script> </object> </blockquote> </pre>
</table> </table> </table> </table> </table> </font> </font> </font>'''
```
Then it works very well, and it has good view.Anyone know what is the situation in ngix?
msg376811 - (view) Author: (igs) Date: 2020-09-12 19:25
As mentioned above standard Apache does not accept the extra characters anymore and produces '500 internal error'. So the normal behaviour of this module makes things worse in most cases instead of being helpful.
msg389031 - (view) Author: coyot linden (coyot) Date: 2021-03-18 17:21
Ran into this also, got:

AH02429: Response header name '<!--' contains invalid characters, aborting request
msg415246 - (view) Author: Colin 't Hart (cthart) Date: 2022-03-15 14:23
1. This module is scheduled to be removed by Python 3.13 (although I preseonally am of the opinion that it is a useful module and would like to see it brought up-to-date).
2. Is reset() even necessary anymore? Can't the same results be achieved with CSS since we are in the third decade of the 2000s after all?
History
Date User Action Args
2022-04-11 14:57:00adminsetgithub: 52950
2022-03-15 14:23:55cthartsetnosy: + cthart
messages: + msg415246
2021-03-18 17:21:55coyotsetnosy: + coyot
messages: + msg389031
2020-09-12 19:25:28igssetnosy: + igs
messages: + msg376811
2020-03-14 15:51:46Ryan Tusetnosy: + Ryan Tu

messages: + msg364186
versions: + Python 3.8, - Python 2.7, Python 3.2, Python 3.3, Python 3.5
2017-05-20 10:40:20Артур Клесунsetnosy: + Артур Клесун

messages: + msg294027
versions: + Python 3.5
2013-07-16 20:51:52p0lar_bearsetversions: + Python 3.3
2013-07-16 20:44:40p0lar_bearsetnosy: + p0lar_bear
messages: + msg193189
2011-03-18 02:11:43orsenthilsetnosy: + orsenthil
2010-05-14 13:06:35stutzbachsetmessages: + msg105705
2010-05-14 07:57:48ysj.raysetnosy: + ysj.ray
messages: + msg105690
2010-05-14 01:44:28meatballhatsetnosy: + meatballhat
2010-05-13 14:12:58stutzbachsetkeywords: + easy
2010-05-13 14:12:50stutzbachcreate