classification
Title: wsgiref.simple_server sends headers on empty bytes
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: berker.peksag, pje, rschoon
Priority: normal Keywords: patch

Created on 2014-07-01 03:15 by rschoon, last changed 2016-05-06 11:45 by berker.peksag.

Files
File name Uploaded Description Edit
wsgiref-empty-byte.patch rschoon, 2014-07-01 03:15 review
wsgiref-empty-byte-2.patch rschoon, 2014-07-02 23:00 hopefully not wrongheaded this time review
wsgiref-empty-byte-3.patch rschoon, 2014-07-02 23:53 add another test review
Messages (5)
msg222005 - (view) Author: Robin Schoonover (rschoon) * Date: 2014-07-01 03:15
Consider this paragraph of PEP3333, referring to headers obtained
via start_response, emphasis mine:

    Instead, it must store them for the server or gateway to
    transmit only after the first iteration of the application
    return value that yields a *non-empty bytestring*, or upon
    the application's first invocation of the write() callable.

This means that an WSGI app such as this should be valid, because the
yielded bytes pre-start_response are empty:

    def application(environ, start_response):
        yield b''
        start_response("200 OK", [("Content-Type", "text/plain")])
        yield b'Hello, World.\n'

However, in wsgiref's simple server, this fails:

    Traceback (most recent call last):
      File "/usr/local/lib/python3.4/wsgiref/handlers.py", line 180, in finish_response
        self.write(data)
      File "/usr/local/lib/python3.4/wsgiref/handlers.py", line 269, in write
        raise AssertionError("write() before start_response()")
    AssertionError: write() before start_response()
msg222117 - (view) Author: PJ Eby (pje) * (Python committer) Date: 2014-07-02 18:57
Please see this paragraph of the spec (my emphasis added):

(Note: the application must invoke the start_response() callable **before the iterable yields its first body string**, so that the server can send the headers before any body content. However, this invocation may be performed by the iterable's first iteration, so servers must not assume that start_response() has been called before they begin iterating over the iterable.)

The paragraph you quoted says that start_response() has to buffer headers until a non-empty string is yielded.  It does *not* say that strings can be yielded prior to calling start_response().  Indeed, the paragraph I quote above states the opposite: you can't call start_response() before yielding your first body string (whether empty or not).

This is a known issue with the spec, but it's an issue with the *spec*, not the implementation.  WSGI 1.0 is known to be unusable as a truly async API, for this and other reasons.
msg222135 - (view) Author: Robin Schoonover (rschoon) * Date: 2014-07-02 21:54
Fair enough, I misled myself.

However, and I feel like I'm getting really picky here, but it still doesn't fulfill the paragraph I quoted:

    def application(environ, start_response):
        start_response('200 OK',
                   [('Content-type', 'text/plain')])
        yield b''

        try:
            # produce an exception tuple, so we can re-call s_r
            raise RuntimeError
        except RuntimeError:
            # Headers shouldn't have been sent, but they were
            # so this will throw:
            start_response('200 OK',
                           [('Content-type', 'text/plain')],
                           sys.exc_info())
        yield b'error data or whatever'

But if async support a foregone conclusion anyway, is it worth bothering complying with that odd requirement?
msg222138 - (view) Author: PJ Eby (pje) * (Python committer) Date: 2014-07-02 22:26
You're right, it shouldn't send the headers until a non-empty string
occurs.  I don't see any problem with treating it as a bug, and fixing it.
Your patch will also allow non-compliant behavior, though.  It seems to me
it would be better to fix the logic in write() to not call send_headers()
if len(data)==0.  That way, it will still error with "write() before
start_response()" in the non-compliant case, but fix the compliance error.
Feel free to reopen/retitle this issue for that.
msg222143 - (view) Author: Robin Schoonover (rschoon) * Date: 2014-07-02 23:00
I agree, the current patch is too permissive.

Both a server I wrote a while ago, and most other "complaint" servers deal with the problem the exact same way as that patch, and that extra permissiveness led to my misinterpretation when analyzing why I had made that original change.

In any case, I've attached an updated patch.
History
Date User Action Args
2016-05-06 11:45:50berker.peksagsetnosy: + berker.peksag
stage: patch review

versions: + Python 2.7, Python 3.5, Python 3.6
2014-07-02 23:53:11rschoonsetfiles: + wsgiref-empty-byte-3.patch
2014-07-02 23:00:36rschoonsetstatus: closed -> open
files: + wsgiref-empty-byte-2.patch
title: wsgiref.simple_server doesn't accept empty bytes before start_response is called -> wsgiref.simple_server sends headers on empty bytes
messages: + msg222143

resolution: not a bug ->
2014-07-02 22:26:54pjesetmessages: + msg222138
2014-07-02 21:54:21rschoonsetmessages: + msg222135
2014-07-02 18:57:06pjesetstatus: open -> closed
resolution: not a bug
messages: + msg222117
2014-07-01 03:15:44rschoonsetfiles: + wsgiref-empty-byte.patch
keywords: + patch
2014-07-01 03:15:10rschooncreate