wsgiref does not call close() on iterable response #60424

btubbs · 2012-10-13T21:00:21Z

BPO	16220
Nosy	@jcea, @pjeby, @pitrou
Files	sleepy_app.py: test script wsgiref_close.patch wsgiref_close_plus_test.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-10-21.12:17:46.643>
created_at = <Date 2012-10-13.21:00:20.662>
labels = ['type-bug', 'library']
title = 'wsgiref does not call close() on iterable response'
updated_at = <Date 2012-10-21.12:17:46.642>
user = 'https://bugs.python.org/btubbs'

bugs.python.org fields:

activity = <Date 2012-10-21.12:17:46.642>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2012-10-21.12:17:46.643>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2012-10-13.21:00:20.662>
creator = 'btubbs'
dependencies = []
files = ['27555', '27561', '27608']
hgrepos = []
issue_num = 16220
keywords = ['patch']
message_count = 13.0
messages = ['172830', '172831', '172841', '172858', '172937', '172939', '172940', '172945', '172952', '173234', '173442', '173445', '173446']
nosy_count = 6.0
nosy_names = ['jcea', 'pje', 'pitrou', 'grahamd', 'python-dev', 'btubbs']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue16220'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4']

btubbs · 2012-10-13T21:00:20Z

When a WSGI application returns an iterable that has a .close() method, the server is supposed to call that method once the request has finished. The wsgiref server does not do this when a client disconnects from a streaming response.

The attached script allows testing the .close() behavior of various wsgi servers (wsgiref, cherrypy, gevent, werkzeug, and gunicorn). wsgiref is the only one of the tested implementations that does not call .close().

pitrou · 2012-10-13T21:14:29Z

This sounds like a reasonable expectation. Do you want to provide a patch?

pjeby · 2012-10-14T00:26:46Z

FYI, this looks like a bug in wsgiref.handlers.BaseHandler.finish_response(), which should probably be using a try/finally to ensure .close() gets called. (Which would catch a failed write() to the client.)

I'm kind of amazed this has gone undetected this long, but I guess that either:

other servers are probably catching errors from .run() and closing,
they're not using BaseHandler in their implementation, or
most apps don't have anything that essential in close() and/or the clients don't disconnect that often. ;-)

btubbs · 2012-10-14T08:41:17Z

Patch attached.

I ran into this while trying to figure out why close() wasn't being called while running the Django dev server, which inherits from wsgiref's BaseHandler. So I'm also surprised that it's gone unnoticed so long.

Thanks for the quick attention and encouragement on this!

jcea · 2012-10-15T02:22:50Z

Brent, could you possibly provide also a test?. Something to integrate with the standard testsuite that fails without your patch but passes with it.

jcea · 2012-10-15T02:30:37Z

Your patch is trivial enough, I am OK to integrate it as is, but it would be nice to verify that we are not reintroducing this bug in the future.

I know that writing a the test will be far more difficult that writing the patch. You can try :).

Anyway I am OK to integrate current patch as is. Looks simple enough.

The comment in "run()" is a bit scary, and we could call "self.close()" twice, with this patch, or call extra code after "self.close()" is called. I don't know it that would be an issue, I am not familiarized enough with WSGI.

grahamd · 2012-10-15T02:40:05Z

Hmmm. Wonders if finally finding this was prompted in part by recent post about this very issue. :-)

http://blog.dscpl.com.au/2012/10/obligations-for-calling-close-on.html

Also related is this issue from Django I highlighted long time ago.

https://code.djangoproject.com/ticket/16241

I would have to look through Django code again but wondering if the issue there was in fact caused by underlying issue in standard library or whether would have needed to be separately fixed in Django still.

btubbs · 2012-10-15T05:19:58Z

You guessed it Graham! Bob Brewer pointed me to your post while I was fighting with this, which led me to testing the behavior under various servers and finding the wsgiref issue.

Current Django trunk doesn't have its own finish_response anymore for the dev server; it's using the one on wsgiref's BaseHandler. So Django's problem should get fixed when wsgiref's does.

I'm working on a test of the simpler case of ensuring that close() is called when any old exception is raised during the response. A test that actually sets up a socket and tests the client disconnecting would be a lot more complicated and a bit out of step with the current wsgi tests that all seem to use a socket-less mock.

grahamd · 2012-10-15T09:11:41Z

That's right, the Django bug report I filed was actually for Django 1.3, which didn't use wsgiref. I wasn't using Django 1.4 at the time so didn't bother to check its new implementation based on wsgiref. Instead I just assumed wsgiref would be right. Whoops.

btubbs · 2012-10-18T06:32:25Z

Updated patch with test attached.

python-dev · 2012-10-21T12:11:11Z

New changeset d5af1b235dab by Antoine Pitrou in branch '2.7':
Issue bpo-16220: wsgiref now always calls close() on an iterable response.
http://hg.python.org/cpython/rev/d5af1b235dab

python-dev · 2012-10-21T12:17:12Z

New changeset eef470032457 by Antoine Pitrou in branch '3.2':
Issue bpo-16220: wsgiref now always calls close() on an iterable response.
http://hg.python.org/cpython/rev/eef470032457

New changeset 2530acc092d8 by Antoine Pitrou in branch '3.3':
Issue bpo-16220: wsgiref now always calls close() on an iterable response.
http://hg.python.org/cpython/rev/2530acc092d8

New changeset cf3a739345c6 by Antoine Pitrou in branch 'default':
Issue bpo-16220: wsgiref now always calls close() on an iterable response.
http://hg.python.org/cpython/rev/cf3a739345c6

pitrou · 2012-10-21T12:17:47Z

Your patch is now committed, Brent, thank you!

btubbs mannequin added the stdlib Python modules in the Lib dir label Oct 13, 2012

btubbs mannequin changed the title ~~wsgiref does not call close() on iterable response when client~~ wsgiref does not call close() on iterable response Oct 13, 2012

pitrou added the type-bug An unexpected behavior, bug, or error label Oct 13, 2012

pitrou closed this as completed Oct 21, 2012

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

wsgiref does not call close() on iterable response #60424

wsgiref does not call close() on iterable response #60424

btubbs mannequin commented Oct 13, 2012

btubbs mannequin commented Oct 13, 2012

pitrou commented Oct 13, 2012

pjeby mannequin commented Oct 14, 2012

btubbs mannequin commented Oct 14, 2012

jcea commented Oct 15, 2012

jcea commented Oct 15, 2012

grahamd mannequin commented Oct 15, 2012

btubbs mannequin commented Oct 15, 2012

grahamd mannequin commented Oct 15, 2012

btubbs mannequin commented Oct 18, 2012

python-dev mannequin commented Oct 21, 2012

python-dev mannequin commented Oct 21, 2012

pitrou commented Oct 21, 2012

wsgiref does not call close() on iterable response #60424

wsgiref does not call close() on iterable response #60424

Comments

btubbs mannequin commented Oct 13, 2012

btubbs mannequin commented Oct 13, 2012

pitrou commented Oct 13, 2012

pjeby mannequin commented Oct 14, 2012

btubbs mannequin commented Oct 14, 2012

jcea commented Oct 15, 2012

jcea commented Oct 15, 2012

grahamd mannequin commented Oct 15, 2012

btubbs mannequin commented Oct 15, 2012

grahamd mannequin commented Oct 15, 2012

btubbs mannequin commented Oct 18, 2012

python-dev mannequin commented Oct 21, 2012

python-dev mannequin commented Oct 21, 2012

pitrou commented Oct 21, 2012