Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

asyncio: recv_into() must not return b'' if the socket/pipe is closed #85639

Closed
vstinner opened this issue Aug 3, 2020 · 7 comments
Closed

asyncio: recv_into() must not return b'' if the socket/pipe is closed #85639

vstinner opened this issue Aug 3, 2020 · 7 comments
Labels
3.10 only security fixes topic-asyncio

Comments

@vstinner
Copy link
Member

vstinner commented Aug 3, 2020

BPO 41467
Nosy @db3l, @vstinner, @asvetlov, @1st1, @miss-islington
PRs
  • bpo-41467: Fix asyncio recv_into() on Windows #21720
  • [3.9] bpo-41467: Fix asyncio recv_into() on Windows (GH-21720) #21724
  • [3.8] bpo-41467: Fix asyncio recv_into() on Windows (GH-21720) #21726
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-08-05.00:09:54.697>
created_at = <Date 2020-08-03.22:19:49.785>
labels = ['3.10', 'expert-asyncio']
title = "asyncio: recv_into() must not return b'' if the socket/pipe is closed"
updated_at = <Date 2020-08-05.18:49:30.857>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2020-08-05.18:49:30.857>
actor = 'db3l'
assignee = 'none'
closed = True
closed_date = <Date 2020-08-05.00:09:54.697>
closer = 'vstinner'
components = ['asyncio']
creation = <Date 2020-08-03.22:19:49.785>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 41467
keywords = ['patch']
message_count = 7.0
messages = ['374760', '374762', '374764', '374778', '374780', '374782', '374895']
nosy_count = 5.0
nosy_names = ['db3l', 'vstinner', 'asvetlov', 'yselivanov', 'miss-islington']
pr_nums = ['21720', '21724', '21726']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue41467'
versions = ['Python 3.10']
    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 3, 2020

    The proactor event loop of asyncio returns b'' on recv_into() if the socket/pipe is closed:

        def recv_into(self, conn, buf, flags=0):
        ...
        try:
            ...
        except BrokenPipeError:
            return self._result(b'')
        ...

    But a socket recv_into() must always return a number: the number of written bytes. Example with socket.socket.recv_into() method:
    https://docs.python.org/3/library/socket.html#socket.socket.recv_into

    IMHO zero must be returned here. This bug may be the root cause of bpo-38912 bug.

    Attached PR fix the issue.

    @vstinner vstinner added 3.10 only security fixes topic-asyncio labels Aug 3, 2020
    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 3, 2020

    The function was added in 2017 in bpo-31819, PR 4051:

    commit 525f40d
    Author: Antoine Pitrou <pitrou@free.fr>
    Date: Thu Oct 19 21:46:40 2017 +0200

    bpo-31819: Add AbstractEventLoop.sock_recv_into() (bpo-4051)

    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 3, 2020

    This bug may be the root cause of bpo-38912 bug.

    Yeah, very likely. This bug makes asyncio inconsistent. A transport is "not closed" and "closed" at the same time...

    C:\vstinner\python\master\lib\asyncio\proactor_events.py:121: ResourceWarning: unclosed transport <_ProactorReadPipeTransport>
    _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
    ResourceWarning: Enable tracemalloc to get the object allocation traceback

    Warning -- Unraisable exception
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001F30C938EB0>
Traceback (most recent call last):
  File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 122, in __del__
    self.close()
  File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 114, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 746, in call_soon
    self._check_closed()
  File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 4, 2020

    New changeset 602a971 by Victor Stinner in branch 'master':
    bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
    602a971

    @miss-islington
    Copy link
    Contributor

    New changeset b934d83 by Miss Islington (bot) in branch '3.8':
    bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
    b934d83

    @miss-islington
    Copy link
    Contributor

    New changeset 1d16229 by Miss Islington (bot) in branch '3.9':
    bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
    1d16229

    @vstinner vstinner closed this as completed Aug 5, 2020
    @vstinner vstinner closed this as completed Aug 5, 2020
    @db3l
    Copy link
    Contributor

    db3l commented Aug 5, 2020

    Just for the record, this fix also appears to have resolved the issue with the Win10 buildbot from bpo-41273, which was related to the same unraisable exceptions mentioned in bpo-38912.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.10 only security fixes topic-asyncio
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @vstinner @db3l @miss-islington