Title: WSGI, cgi.FieldStorage incompatibility
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: dstanek, eric.araujo, exarkun, flox, jcea, mcdonc, mkerrin, orsenthil
Priority: normal Keywords: patch

Created on 2006-10-09 18:34 by mkerrin, last changed 2014-02-10 14:08 by jcea.

File name Uploaded Description Edit
cgi.patch mkerrin, 2006-10-09 18:34 review
Messages (5)
msg30196 - (view) Author: Michael Kerrin (mkerrin) Date: 2006-10-09 18:34
The WSGI specification says in the section on "Input
and Error Streams":

  The optional "size" argument to readline() is not
supported, as it may be complex for server authors to
implement, and is not often used in practice.

  But the current implementation of cgi.FieldStorage in
the 2.4.4 branch and on Python 2.5 does call readline
with the size argument. It has started to do this in
response to the Python bug #1112549 - cgi.FieldStorage
memory usage can spike in line-oriented ops. See
  I am not 100% sure wheather this is a bug in the WSGI
application or cgi.FieldStorage class,

  Originally I thought it was cgi.FieldStorage that
needs to be changed, and hence tried to fix it by
wrapping the input stream so that the readline method
always uses the read method on the input stream (so has
to not to break the above mentioned bug). While this
seems to work for me it introduces a level of
complexity in the file, and possible some other
bugs, that makes me think that adding the size argument
for readline into the WSGI specification isn't such bad
idea after all.

  I have attached my patch against cgi.FieldStorage if
any one thinks this is the best way of fixing this issue.

  This is an issue for Zope3 has Zope3 doesn't support
the size argument and neither does Twisted from which
Zope gets its input stream.
msg108181 - (view) Author: Chris McDonough (mcdonc) Date: 2010-06-19 13:50
<Review prompted by "500 tracker orphans; need more reviewers">

Although the current WSGI 1.0 spec ( still indicates that implementing the size argument to readline is not required, effectively, it is and has been for several years.  Stalling on a WSGI 1.1 spec has caused a spec change to be delayed.  See also
msg108182 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2010-06-19 13:57
It's not terribly productive to block a fix for this specific issue in the WSGI specification on the big pile of contentious unrelated issues.

It would make sense to issue a new WSGI specification with a correction for only this issue.  The rest of the WSGI 1.1 issues can wait for a subsequent revision.

Of course, actually getting this done involves either getting web-sig behind it or deciding to ignore web-sig and just do it.
msg110502 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-07-16 21:30
Practicality beats purity?  The patch contains an enhancement to the unit test for cgi.  Provided this gets past regrtest on all platforms why can't we go for it?
msg139648 - (view) Author: √Čric Araujo (eric.araujo) * (Python committer) Date: 2011-07-02 13:52
Now that web-sig has reached agreement on PEP 3333, can an expert update the status of this bug?
Date User Action Args
2014-02-10 14:08:26jceasetnosy: + jcea
2014-02-03 19:10:30BreamoreBoysetnosy: - BreamoreBoy
2012-07-21 14:13:00floxsetnosy: + orsenthil

versions: + Python 3.3, - Python 3.1
2011-07-02 13:52:29eric.araujosetnosy: + eric.araujo
messages: + msg139648
2011-07-02 13:41:13eric.araujolinkissue6234 superseder
2010-08-27 03:14:26floxsetnosy: + flox

versions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6
2010-08-04 10:54:51dstaneksetnosy: + dstanek
2010-07-16 21:30:28BreamoreBoysetnosy: + BreamoreBoy
messages: + msg110502
2010-06-19 13:57:33exarkunsetnosy: + exarkun
messages: + msg108182
2010-06-19 13:50:04mcdoncsetnosy: + mcdonc
messages: + msg108181
2009-03-30 07:21:29ajaksu2setkeywords: + patch
stage: patch review
type: behavior
components: + Library (Lib), - None
versions: + Python 2.6, - Python 2.4
2006-10-09 18:34:30mkerrincreate