classification
Title: asyncio: bug in _ProactorBasePipeTransport._force_close()
Type: Stage: resolved
Components: asyncio, Windows Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, miss-islington, paul.moore, steve.dower, tim.golden, vstinner, yselivanov, zach.ware
Priority: normal Keywords: patch, patch, patch, patch

Created on 2019-01-08 00:50 by vstinner, last changed 2019-01-08 02:15 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 11462 merged vstinner, 2019-01-08 00:55
PR 11462 merged vstinner, 2019-01-08 00:55
PR 11462 merged vstinner, 2019-01-08 00:55
PR 11462 merged vstinner, 2019-01-08 00:55
PR 11463 merged miss-islington, 2019-01-08 01:54
PR 11463 merged miss-islington, 2019-01-08 01:54
PR 11463 merged miss-islington, 2019-01-08 01:54
PR 11463 merged miss-islington, 2019-01-08 01:54
Messages (3)
msg333192 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-01-08 00:50
Running ProactorEventLoopTests.test_sendfile_close_peer_in_the_middle_of_receiving() logs a bug in _force_close(): see logs below. Extract of _force_close():

    def _force_close(self, exc):
        if self._empty_waiter is not None:
            if exc is None:
                self._empty_waiter.set_result(None)
            else:
                self._empty_waiter.set_exception(exc)
        ...

Problem: _empty_waiter can be already done. For example, it can be created directly as done:

    def _make_empty_waiter(self):
        ...
        self._empty_waiter = self._loop.create_future()
        if self._write_fut is None:
            self._empty_waiter.set_result(None)
        return self._empty_waiter

Attached PR fixes _force_close(): do nothing if _empty_waiter is already done.

The regression comes from the following change:

commit a19fb3c6aaa7632410d1d9dcb395d7101d124da4
Author: Andrew Svetlov <andrew.svetlov@gmail.com>
Date:   Sun Feb 25 19:32:14 2018 +0300

    bpo-32622: Native sendfile on windows (#5565)
    
    * Support sendfile on Windows Proactor event loop naively.


Logs:

vstinner@WIN C:\vstinner\python\master>python -X dev -m test test_asyncio -m test.test_asyncio.test_sendfile.ProactorEventLoopTests.test_sendfile_close_peer_in_the_middle_of_receiving
Running Debug|x64 interpreter...
Run tests sequentially 
0:00:00 [1/1] test_asyncio 
Exception in callback _ProactorReadPipeTransport._loop_reading(<_OverlappedF...events.py:452>) 
handle: <Handle _ProactorReadPipeTransport._loop_reading(<_OverlappedF...events.py:452>) created at C:\vstinner\python\master\lib\asyncio\windows_events.py:82>
source_traceback: Object created at (most recent call last):
  File "C:\vstinner\python\master\lib\test\test_asyncio\test_sendfile.py", line 125, in run_loop
    return self.loop.run_until_complete(coro)
  File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 576, in run_until_complete
    self.run_forever()
  File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 315, in run_forever
    super().run_forever()
  File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 544, in run_forever
    self._run_once()
  File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 1729, in _run_once
    event_list = self._selector.select(timeout)
  File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 421, in select
    self._poll(timeout)
  File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 750, in _poll
    f.set_exception(e)
  File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 82, in set_exception
    super().set_exception(exception)
Traceback (most recent call last):
  File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 444, in finish_recv
    return ov.getresult()
OSError: [WinError 64] The specified network name is no longer available

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 256, in _loop_reading
    data = fut.result()
  File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 748, in _poll
    value = callback(transferred, key, ov)
  File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 448, in finish_recv
    raise ConnectionResetError(*exc.args)
ConnectionResetError: [WinError 64] The specified network name is no longer available

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\vstinner\python\master\lib\asyncio\events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 283, in _loop_reading
    self._force_close(exc)
  File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 118, in _force_close
    self._empty_waiter.set_exception(exc)
asyncio.exceptions.InvalidStateError: invalid state

== Tests result: SUCCESS ==

1 test OK.

Total duration: 531 ms
Tests result: SUCCESS
msg333197 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-01-08 01:47
New changeset 80fda712c83f5dd9560d42bf2aa65a72b18b7759 by Victor Stinner in branch 'master':
bpo-35682: Fix _ProactorBasePipeTransport._force_close() (GH-11462)
https://github.com/python/cpython/commit/80fda712c83f5dd9560d42bf2aa65a72b18b7759
msg333202 - (view) Author: miss-islington (miss-islington) Date: 2019-01-08 02:15
New changeset 88bd26a72eb4ab341cf19bea78a0039fbe4be3a2 by Miss Islington (bot) in branch '3.7':
bpo-35682: Fix _ProactorBasePipeTransport._force_close() (GH-11462)
https://github.com/python/cpython/commit/88bd26a72eb4ab341cf19bea78a0039fbe4be3a2
History
Date User Action Args
2019-01-08 15:16:18vstinnerlinkissue33834 superseder
2019-01-08 02:15:57vstinnersetkeywords: patch, patch, patch, patch
status: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-01-08 02:15:36miss-islingtonsetnosy: + miss-islington
messages: + msg333202
2019-01-08 01:55:32miss-islingtonsetpull_requests: + pull_request10948
2019-01-08 01:55:21miss-islingtonsetpull_requests: + pull_request10947
2019-01-08 01:55:06miss-islingtonsetpull_requests: + pull_request10946
2019-01-08 01:54:51miss-islingtonsetpull_requests: + pull_request10945
2019-01-08 01:47:23vstinnersetmessages: + msg333197
2019-01-08 00:56:24vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request10942
2019-01-08 00:56:10vstinnersetkeywords: + patch
stage: (no value)
pull_requests: + pull_request10943
2019-01-08 00:55:55vstinnersetkeywords: + patch
stage: (no value)
pull_requests: + pull_request10941
2019-01-08 00:55:40vstinnersetkeywords: + patch
stage: (no value)
pull_requests: + pull_request10940
2019-01-08 00:50:48vstinnercreate