classification
Title: ProactorEventLoop raises AssertionError
Type: behavior Stage: resolved
Components: asyncio Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, miss-islington, twisteroid ambassador, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2018-06-11 13:47 by twisteroid ambassador, last changed 2018-07-30 22:13 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
test_proactor_force_close.py twisteroid ambassador, 2018-06-11 13:47
Pull Requests
URL Status Linked Edit
PR 7632 closed twisteroid ambassador, 2018-06-11 13:54
PR 7893 merged twisteroid ambassador, 2018-06-24 12:21
PR 8575 merged miss-islington, 2018-07-30 18:59
Messages (8)
msg319302 - (view) Author: twisteroid ambassador (twisteroid ambassador) * Date: 2018-06-11 13:47
Sometimes when a socket transport under ProactorEventLoop is writing while the peer closes the connection, asyncio logs an AssertionError. 

Attached is a script that fairly reliably reproduces the behavior on my computer.

This is caused by _ProactorBasePipeTransport._force_close() being called between two invocations of _ProactorBaseWritePipeTransport._loop_writing(), where the latter call asserts self._write_fut has not changed after being set by the former call.
msg319306 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-06-11 14:46
Adding Victor as he's been helping with asyncio/proactor lately.
msg320584 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-27 14:27
> Sometimes when a socket transport under ProactorEventLoop is writing while the peer closes the connection, asyncio logs an AssertionError. 

Would you mind to add an example of such error in this issue?
msg320642 - (view) Author: twisteroid ambassador (twisteroid ambassador) * Date: 2018-06-28 02:10
No problem. Running the attached test script on latest master, Windows 10 1803, several errors like this are logged:

Exception in callback _ProactorBaseWritePipeTransport._loop_writing(<_OverlappedF...events.py:479>)
handle: <Handle _ProactorBaseWritePipeTransport._loop_writing(<_OverlappedF...events.py:479>) created at %USERPROFILE%\source\repos\cpython\lib\asyncio\proactor_events.py:373>
source_traceback: Object created at (most recent call last):
  File "%USERPROFILE%\source\repos\cpython\lib\asyncio\base_events.py", line 555, in run_until_complete
    self.run_forever()
  File "%USERPROFILE%\source\repos\cpython\lib\asyncio\base_events.py", line 523, in run_forever
    self._run_once()
  File "%USERPROFILE%\source\repos\cpython\lib\asyncio\base_events.py", line 1750, in _run_once
    handle._run()
  File "%USERPROFILE%\source\repos\cpython\lib\asyncio\events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "..\..\..\..\Documents\python\test_proactor_force_close.py", line 8, in server_callback
    writer.write(b'hello')
  File "%USERPROFILE%\source\repos\cpython\lib\asyncio\streams.py", line 305, in write
    self._transport.write(data)
  File "%USERPROFILE%\source\repos\cpython\lib\asyncio\proactor_events.py", line 334, in write
    self._loop_writing(data=bytes(data))
  File "%USERPROFILE%\source\repos\cpython\lib\asyncio\proactor_events.py", line 373, in _loop_writing
    self._write_fut.add_done_callback(self._loop_writing)
Traceback (most recent call last):
  File "%USERPROFILE%\source\repos\cpython\lib\asyncio\events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "%USERPROFILE%\source\repos\cpython\lib\asyncio\proactor_events.py", line 346, in _loop_writing
    assert f is self._write_fut
AssertionError
msg320643 - (view) Author: twisteroid ambassador (twisteroid ambassador) * Date: 2018-06-28 02:18
As an aside, I'm wondering whether it makes sense to add a blanket "assert exception handler has not been called" condition to ProactorEventLoop's tests, or even other asyncio tests. It looks like ProactorEventLoop is more prone to dropping exceptions on the floor than SelectorEventLoop.
msg322685 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2018-07-30 18:58
New changeset 9045199c5aaeac9b52537581be127d999b5944ee by Andrew Svetlov (twisteroid ambassador) in branch 'master':
bpo-33833: Fix ProactorSocketTransport AssertionError (#7893)
https://github.com/python/cpython/commit/9045199c5aaeac9b52537581be127d999b5944ee
msg322687 - (view) Author: miss-islington (miss-islington) Date: 2018-07-30 20:04
New changeset d5c75be55504fae1ff510eed66cddbd27bfbdbe2 by Miss Islington (bot) in branch '3.7':
bpo-33833: Fix ProactorSocketTransport AssertionError (GH-7893)
https://github.com/python/cpython/commit/d5c75be55504fae1ff510eed66cddbd27bfbdbe2
msg322693 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-30 22:13
Why not fixing Python 3.6? It is not affected?
History
Date User Action Args
2018-07-30 22:13:58vstinnersetmessages: + msg322693
2018-07-30 20:14:24asvetlovsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-07-30 20:04:34miss-islingtonsetnosy: + miss-islington
messages: + msg322687
2018-07-30 18:59:02miss-islingtonsetpull_requests: + pull_request8082
2018-07-30 18:58:54asvetlovsetmessages: + msg322685
2018-06-28 02:18:35twisteroid ambassadorsetmessages: + msg320643
2018-06-28 02:10:45twisteroid ambassadorsetmessages: + msg320642
2018-06-27 14:27:05vstinnersetmessages: + msg320584
2018-06-24 12:21:48twisteroid ambassadorsetpull_requests: + pull_request7500
2018-06-11 14:46:44yselivanovsetnosy: + vstinner
messages: + msg319306
2018-06-11 13:54:02twisteroid ambassadorsetkeywords: + patch
stage: patch review
pull_requests: + pull_request7251
2018-06-11 13:47:44twisteroid ambassadorsettype: behavior
2018-06-11 13:47:30twisteroid ambassadorcreate