classification
Title: asyncio: recv_into() must not return b'' if the socket/pipe is closed
Type: Stage: resolved
Components: asyncio Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, db3l, miss-islington, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2020-08-03 22:19 by vstinner, last changed 2020-08-05 18:49 by db3l. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 21720 merged vstinner, 2020-08-03 22:21
PR 21724 merged miss-islington, 2020-08-04 00:40
PR 21726 merged miss-islington, 2020-08-04 00:41
Messages (7)
msg374760 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-03 22:19
The proactor event loop of asyncio returns b'' on recv_into() if the socket/pipe is closed:

    def recv_into(self, conn, buf, flags=0):
        ...
        try:
            ...
        except BrokenPipeError:
            return self._result(b'')
        ...

But a socket recv_into() must always return a number: the number of written bytes. Example with socket.socket.recv_into() method:
https://docs.python.org/3/library/socket.html#socket.socket.recv_into

IMHO zero must be returned here. This bug may be the root cause of bpo-38912 bug.

Attached PR fix the issue.
msg374762 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-03 22:35
The function was added in 2017 in bpo-31819, PR 4051:

commit 525f40d231aba2c004619fc7a5207171ed65b0cb
Author: Antoine Pitrou <pitrou@free.fr>
Date:   Thu Oct 19 21:46:40 2017 +0200

    bpo-31819: Add AbstractEventLoop.sock_recv_into() (#4051)
msg374764 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-03 22:44
> This bug may be the root cause of bpo-38912 bug.

Yeah, very likely. This bug makes asyncio inconsistent. A transport is "not closed" and "closed" at the same time...

C:\vstinner\python\master\lib\asyncio\proactor_events.py:121: ResourceWarning: unclosed transport <_ProactorReadPipeTransport>
  _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback

Warning -- Unraisable exception
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001F30C938EB0>
Traceback (most recent call last):
  File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 122, in __del__
    self.close()
  File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 114, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 746, in call_soon
    self._check_closed()
  File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
msg374778 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-04 00:40
New changeset 602a971a2af3a685d625c912c400cadd452718b1 by Victor Stinner in branch 'master':
bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
https://github.com/python/cpython/commit/602a971a2af3a685d625c912c400cadd452718b1
msg374780 - (view) Author: miss-islington (miss-islington) Date: 2020-08-04 00:58
New changeset b934d832d1e16bf235c536dcde3006faf29757fc by Miss Islington (bot) in branch '3.8':
bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
https://github.com/python/cpython/commit/b934d832d1e16bf235c536dcde3006faf29757fc
msg374782 - (view) Author: miss-islington (miss-islington) Date: 2020-08-04 01:00
New changeset 1d16229f3f5b91f2389c7c5c6425c5524c413651 by Miss Islington (bot) in branch '3.9':
bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
https://github.com/python/cpython/commit/1d16229f3f5b91f2389c7c5c6425c5524c413651
msg374895 - (view) Author: David Bolen (db3l) * Date: 2020-08-05 18:49
Just for the record, this fix also appears to have resolved the issue with the Win10 buildbot from bpo-41273, which was related to the same unraisable exceptions mentioned in bpo-38912.
History
Date User Action Args
2020-08-05 18:49:30db3lsetnosy: + db3l
messages: + msg374895
2020-08-05 00:09:54vstinnersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-08-04 01:00:36miss-islingtonsetmessages: + msg374782
2020-08-04 00:58:08miss-islingtonsetmessages: + msg374780
2020-08-04 00:41:17miss-islingtonsetpull_requests: + pull_request20869
2020-08-04 00:40:33miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request20867
2020-08-04 00:40:13vstinnersetmessages: + msg374778
2020-08-03 22:44:45vstinnersetmessages: + msg374764
2020-08-03 22:35:29vstinnersetmessages: + msg374762
2020-08-03 22:21:51vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request20863
2020-08-03 22:19:49vstinnercreate