classification
Title: wsgiref.simple_server doesn't handle multi-line headers correctly
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: Alan.Braithwaite, abraithwaite, iritkatriel, pitrou, pje, serhiy.storchaka, tanner
Priority: normal Keywords: patch

Created on 2014-03-31 17:02 by Alan.Braithwaite, last changed 2020-11-30 19:55 by iritkatriel. This issue is now closed.

Files
File name Uploaded Description Edit
simple_server.py.patch Alan.Braithwaite, 2014-03-31 17:02 Expected Behavior
wsgi.diff tanner, 2014-12-09 15:58
wsgi2.diff tanner, 2014-12-10 12:39
Messages (12)
msg215251 - (view) Author: Alan Braithwaite (Alan.Braithwaite) Date: 2014-03-31 17:02
At wsgiref.simple_server:99, it uses the wrong field to get the headers from the request.  In rfc822.Message.readheaders it describes that the headers attribute is an uninterrupted list of the lines in the header.  The dict attribute of Message should contain the proper key/value map of the headers.

I've attached a patch of what I think the behavior should be.  This has incidentally been fixed in python 3+.
msg231373 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-11-19 11:47
Could you provide tests?
msg231736 - (view) Author: Tom Tanner (tanner) * Date: 2014-11-26 23:57
I confirm this fixes a bug.
To reproduce it just add, e.g.

X-Multline-Header: foo
 bar

to your request.
 
It has been fixed in Python 3 with
https://github.com/python/cpython/commit/67dcb80f6e102622e4aa888930d3017fed9834de
msg232145 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-12-04 17:58
Thank you Tom. This was made in issue4718. But tests are passed without them. So 3.x needs tests too.
msg232384 - (view) Author: Tom Tanner (tanner) * Date: 2014-12-09 15:58
This is actually more complicated than I initially thought.
According to https://www.python.org/dev/peps/pep-3333/#the-start-response-callable
"Each header_value must not include any control characters, including carriage returns or linefeeds, either embedded or at the end. (These requirements are to minimize the complexity of any parsing that must be performed by servers, gateways, and intermediate response processors that need to inspect or modify response headers.) "

There's a discussion at https://mail.python.org/pipermail/web-sig/2006-September/002244.html 

It think the spec should be changed as parsing multi-line headers is really simple and sometimes necessary (e.g., when passing the X.509 client certificate).

Anyway, attached you find a patch for Python 2.7.8 which includes a test and allows CR/LF in header values.
msg232399 - (view) Author: PJ Eby (pje) * (Python committer) Date: 2014-12-09 18:50
The fix for the server looks ok, but the validation and tests must not
be changed, since they void spec compatibility and aren't a bug fix
(and so must not be added to 2.7.)  Indeed, if the validation library
*doesn't* fail on CRLF, then *that* would be a bug, since it's
supposed to tell you about every verifiable spec violation.

Also, simple_server *should* unfold multiline headers, if it's not
already doing so.  (If it's not, it is a bug and can therefore be
fixed in 2.7.)

On Tue, Dec 9, 2014 at 10:58 AM, Tom Tanner <report@bugs.python.org> wrote:
>
> Tom Tanner added the comment:
>
> This is actually more complicated than I initially thought.
> According to https://www.python.org/dev/peps/pep-3333/#the-start-response-callable
> "Each header_value must not include any control characters, including carriage returns or linefeeds, either embedded or at the end. (These requirements are to minimize the complexity of any parsing that must be performed by servers, gateways, and intermediate response processors that need to inspect or modify response headers.) "
>
> There's a discussion at https://mail.python.org/pipermail/web-sig/2006-September/002244.html
>
> It think the spec should be changed as parsing multi-line headers is really simple and sometimes necessary (e.g., when passing the X.509 client certificate).
>
> Anyway, attached you find a patch for Python 2.7.8 which includes a test and allows CR/LF in header values.
>
> ----------
> Added file: http://bugs.python.org/file37401/wsgi.diff
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue21114>
> _______________________________________
msg232423 - (view) Author: Tom Tanner (tanner) * Date: 2014-12-10 12:39
attached is the updated patch, which unfolds multiline headers but not validate them (tests included).
msg233968 - (view) Author: Tom Tanner (tanner) * Date: 2015-01-13 21:02
ping
msg240123 - (view) Author: Tom Tanner (tanner) * Date: 2015-04-05 15:57
Any chance to get this into 2.7.10?
msg243934 - (view) Author: Tom Tanner (tanner) * Date: 2015-05-23 17:27
The patch is waiting for inclusion in 2.7.10 :/
msg304521 - (view) Author: (abraithwaite) Date: 2017-10-17 19:50
This might have been fixed by https://bugs.python.org/issue22928

Have not tested.
msg382187 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-11-30 19:55
Python 2-only issue.
History
Date User Action Args
2020-11-30 19:55:34iritkatrielsetstatus: open -> closed

nosy: + iritkatriel
messages: + msg382187

resolution: out of date
stage: test needed -> resolved
2017-10-17 19:50:19abraithwaitesetnosy: + abraithwaite
messages: + msg304521
2015-05-23 17:27:18tannersetmessages: + msg243934
2015-04-05 15:57:37tannersetmessages: + msg240123
2015-01-13 21:02:35tannersetmessages: + msg233968
2014-12-10 12:39:08tannersetfiles: + wsgi2.diff

messages: + msg232423
2014-12-09 18:50:05pjesetmessages: + msg232399
2014-12-09 15:58:22tannersetfiles: + wsgi.diff

messages: + msg232384
2014-12-04 17:58:38serhiy.storchakasetassignee: serhiy.storchaka ->

messages: + msg232145
nosy: + pitrou
2014-11-26 23:57:05tannersetnosy: + tanner
messages: + msg231736
2014-11-19 11:47:28serhiy.storchakasetnosy: + serhiy.storchaka, pje
messages: + msg231373

assignee: serhiy.storchaka
stage: test needed
2014-03-31 17:02:31Alan.Braithwaitecreate