classification
Title: Wsgiref server does not handle closed connections gracefully
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: duplicate
Dependencies: Superseder: wsgiref BaseHandler / SimpleHandler can raise additional errors when handling an error
View: 27682
Assigned To: Nosy List: Petter S, chris.jerdonek, xtreak
Priority: normal Keywords: patch

Created on 2018-08-30 10:17 by Petter S, last changed 2018-11-02 23:08 by chris.jerdonek. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 9713 open Petter S, 2018-10-05 14:08
PR 10090 merged Petter S, 2018-10-27 07:39
Messages (3)
msg324375 - (view) Author: Petter S (Petter S) * Date: 2018-08-30 10:17
The server in the wsgiref module is actually used a lot. For example, it is the server Django uses for development.

A very common thing that happens during Django development is that the web browser closes the connection for some reason. Then very long stack traces appear in the Django logs:

    [30/Aug/2018 12:10:38] "POST /login/ HTTP/1.1" 200 3964
    Traceback (most recent call last):
      File "d:\python37\Lib\wsgiref\handlers.py", line 138, in run
        self.finish_response()
      File "d:\python37\Lib\wsgiref\handlers.py", line 180, in finish_response
        self.write(data)
      File "d:\python37\Lib\wsgiref\handlers.py", line 274, in write
        self.send_headers()
      File "d:\python37\Lib\wsgiref\handlers.py", line 332, in send_headers
        self.send_preamble()
      File "d:\python37\Lib\wsgiref\handlers.py", line 255, in send_preamble
        ('Date: %s\r\n' % format_date_time(time.time())).encode('iso-8859-1')
      File "d:\python37\Lib\wsgiref\handlers.py", line 453, in _write
        result = self.stdout.write(data)
      File "d:\python37\Lib\socketserver.py", line 796, in write
        self._sock.sendall(b)
    ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
    [30/Aug/2018 12:10:38] "POST /login/ HTTP/1.1" 500 59
    ----------------------------------------
    Exception happened during processing of request from ('127.0.0.1', 50112)
    Traceback (most recent call last):
      File "d:\python37\Lib\wsgiref\handlers.py", line 138, in run
        self.finish_response()
      File "d:\python37\Lib\wsgiref\handlers.py", line 180, in finish_response
        self.write(data)
      File "d:\python37\Lib\wsgiref\handlers.py", line 274, in write
        self.send_headers()
      File "d:\python37\Lib\wsgiref\handlers.py", line 332, in send_headers
        self.send_preamble()
      File "d:\python37\Lib\wsgiref\handlers.py", line 255, in send_preamble
        ('Date: %s\r\n' % format_date_time(time.time())).encode('iso-8859-1')
      File "d:\python37\Lib\wsgiref\handlers.py", line 453, in _write
        result = self.stdout.write(data)
      File "d:\python37\Lib\socketserver.py", line 796, in write
        self._sock.sendall(b)
    ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "d:\python37\Lib\wsgiref\handlers.py", line 141, in run
        self.handle_error()
      File "D:\Virtualenvs\ledev-X6wd5Q8f\lib\site-packages\django\core\servers\basehttp.py", line 86, in handle_error
        super().handle_error()
      File "d:\python37\Lib\wsgiref\handlers.py", line 368, in handle_error
        self.finish_response()
      File "d:\python37\Lib\wsgiref\handlers.py", line 180, in finish_response
        self.write(data)
      File "d:\python37\Lib\wsgiref\handlers.py", line 274, in write
        self.send_headers()
      File "d:\python37\Lib\wsgiref\handlers.py", line 331, in send_headers
        if not self.origin_server or self.client_is_modern():
      File "d:\python37\Lib\wsgiref\handlers.py", line 344, in client_is_modern
        return self.environ['SERVER_PROTOCOL'].upper() != 'HTTP/0.9'
    TypeError: 'NoneType' object is not subscriptable

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "d:\python37\Lib\socketserver.py", line 647, in process_request_thread
        self.finish_request(request, client_address)
      File "d:\python37\Lib\socketserver.py", line 357, in finish_request
        self.RequestHandlerClass(request, client_address, self)
      File "d:\python37\Lib\socketserver.py", line 717, in __init__
        self.handle()
      File "D:\Virtualenvs\ledev-X6wd5Q8f\lib\site-packages\django\core\servers\basehttp.py", line 154, in handle
        handler.run(self.server.get_app())
      File "d:\python37\Lib\wsgiref\handlers.py", line 144, in run
        self.close()
      File "d:\python37\Lib\wsgiref\simple_server.py", line 35, in close
        self.status.split(' ',1)[0], self.bytes_sent
    AttributeError: 'NoneType' object has no attribute 'split'
    ----------------------------------------


Obviously, this is a bit annoying when developing with Django daily. Since Django simply uses the wsgiref server, I though the best solution was to handle closed connections more gracefully there.

I think the best solution is to simply add another except clause here: https://github.com/python/cpython/blob/e6dac0077996b1e1f886f036d6f2606237fa4c85/Lib/wsgiref/handlers.py#L142 . We should catch ConnectionAbortedError here and return from the function.

I am happy to create a PR if this is what we want to do.
msg327563 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2018-10-12 06:30
Thanks for the report and PR. I think this is the same as issue27682. I noticed the same during running `make serve` locally for documentation and it's worth fixing. I am just wondering if it makes sense to fix the TypeError and AttributeError too. Also I couldn't see any tests for this and I don't know if it's possible to test this scenario.

I will leave it to the reviewer to close this and raise PR against issue27682 or continue discussion here.

Thanks again!
msg329170 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2018-11-02 23:08
Closing this as a duplicate. Please carry over to issue 27682 any PR's that are still current.
History
Date User Action Args
2018-11-02 23:08:33chris.jerdoneksetstatus: open -> closed

superseder: wsgiref BaseHandler / SimpleHandler can raise additional errors when handling an error

nosy: + chris.jerdonek
messages: + msg329170
resolution: duplicate
stage: patch review -> resolved
2018-10-27 07:39:37Petter Ssetpull_requests: + pull_request9475
2018-10-26 06:35:31Petter Ssetpull_requests: - pull_request9438
2018-10-26 06:35:22Petter Ssetpull_requests: - pull_request9437
2018-10-25 21:16:17Petter Ssetpull_requests: + pull_request9438
2018-10-25 21:16:17Petter Ssetpull_requests: + pull_request9437
2018-10-25 08:12:04Petter Ssetpull_requests: - pull_request9419
2018-10-25 08:02:27Petter Ssetpull_requests: + pull_request9419
2018-10-12 06:30:54xtreaksetmessages: + msg327563
2018-10-12 06:08:48xtreaksetnosy: + xtreak
2018-10-05 14:08:19Petter Ssetkeywords: + patch
stage: patch review
pull_requests: + pull_request9096
2018-08-30 10:17:31Petter Ssettype: enhancement
2018-08-30 10:17:15Petter Screate