Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exceptions raised by EventLoop.call_soon_threadsafe #83832

Closed
bdarnell mannequin opened this issue Feb 16, 2020 · 10 comments
Closed

Exceptions raised by EventLoop.call_soon_threadsafe #83832

bdarnell mannequin opened this issue Feb 16, 2020 · 10 comments
Labels
3.8 only security fixes 3.9 only security fixes 3.10 only security fixes release-blocker topic-asyncio

Comments

@bdarnell
Copy link
Mannequin

bdarnell mannequin commented Feb 16, 2020

BPO 39651
Nosy @vstinner, @asvetlov, @ambv, @bdarnell, @1st1, @cmeyer, @miss-islington, @mikeshardmind
PRs
  • bpo-39651: Fix asyncio proactor _write_to_self() #22197
  • [3.8] bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197) #22215
  • [3.9] bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197) #22216
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-09-12.06:53:41.452>
created_at = <Date 2020-02-16.16:41:58.945>
labels = ['release-blocker', '3.8', '3.9', '3.10', 'expert-asyncio']
title = 'Exceptions raised by EventLoop.call_soon_threadsafe'
updated_at = <Date 2020-09-12.07:11:42.380>
user = 'https://github.com/bdarnell'

    bugs.python.org fields:

    activity = <Date 2020-09-12.07:11:42.380>
actor = 'miss-islington'
assignee = 'none'
closed = True
closed_date = <Date 2020-09-12.06:53:41.452>
closer = 'vstinner'
components = ['asyncio']
creation = <Date 2020-02-16.16:41:58.945>
creator = 'Ben.Darnell'
dependencies = []
files = []
hgrepos = []
issue_num = 39651
keywords = ['patch']
message_count = 10.0
messages = ['362078', '366793', '366839', '366840', '368791', '373594', '376784', '376785', '376790', '376791']
nosy_count = 8.0
nosy_names = ['vstinner', 'asvetlov', 'lukasz.langa', 'Ben.Darnell', 'yselivanov', 'cmeyer', 'miss-islington', 'mikeshardmind']
pr_nums = ['22197', '22215', '22216']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue39651'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']
    @bdarnell
    Copy link
    Mannequin Author

    bdarnell mannequin commented Feb 16, 2020

    Proactor and selector event loops behave differently when call_soon_threadsafe races with a concurrent call to loop.close(). In a selector event loop, call_soon_threadsafe will either succeed or raise a RuntimeError("Event loop is closed"). In a proactor event loop, it could raise this RuntimeError, but it can also raise an AttributeError due to an unguarded access to self._csock.

    cpython/Lib/asyncio/proactor_events.py

    Lines 785 to 787 in 1ed6161

    def _write_to_self(self):
    try:
    self._csock.send(b'\0')

    Comments in BaseSelectorEventLoop._write_to_self indicate that this is deliberate, so the csock is not None check here should probably be copied to the proactor event loop version.

    cpython/Lib/asyncio/selector_events.py

    Lines 129 to 136 in 1ed6161

    def _write_to_self(self):
    # This may be called from a different thread, possibly after
    # _close_self_pipe() has been called or even while it is
    # running. Guard for self._csock being None or closed. When
    # a socket is closed, send() raises OSError (with errno set to
    # EBADF, but let's not rely on the exact error code).
    csock = self._csock
    if csock is not None:

    I'd also accept an answer that the exact behavior of this race is undefined and it's up to the application to either arrange for all calls to call_soon_threadsafe to stop before closing the loop. However, I've had users of Tornado argue that they use the equivalent of call_soon_threadsafe in contexts where this coordination would be difficult, and I've decided that tornado's version of this method would never raise, even if there is a concurrent close. So if asyncio declines to specify which exceptions are allowed in this case, tornado will need to add a blanket `except Exception:` around calls to call_soon_threadsafe.

    @bdarnell bdarnell mannequin added 3.8 only security fixes topic-asyncio labels Feb 16, 2020
    @ambv
    Copy link
    Contributor

    ambv commented Apr 19, 2020

    Good catch. We should fix this for Python 3.8.3.

    @cmeyer
    Copy link
    Mannequin

    cmeyer mannequin commented Apr 20, 2020

    Is this related to bpo-39010 too?

    @bdarnell
    Copy link
    Mannequin Author

    bdarnell mannequin commented Apr 20, 2020

    No, this is unrelated to bpo-39010.

    @ambv
    Copy link
    Contributor

    ambv commented May 13, 2020

    This sadly missed 3.8.3 but I want this addressed for 3.8.4.

    @ambv
    Copy link
    Contributor

    ambv commented Jul 13, 2020

    We have trouble finding a Windows expert with available time to address this :/ This is missing 3.8.4 as well.

    @vstinner
    Copy link
    Member

    New changeset 1b0f0e3 by Victor Stinner in branch 'master':
    bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197)
    1b0f0e3

    @vstinner
    Copy link
    Member

    Thanks for the bug report Ben Darnell, it's now fixed.

    @vstinner vstinner added 3.9 only security fixes 3.10 only security fixes labels Sep 12, 2020
    @miss-islington
    Copy link
    Contributor

    New changeset 530d110 by Miss Islington (bot) in branch '3.8':
    bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197)
    530d110

    @miss-islington
    Copy link
    Contributor

    New changeset 7dfcc8e by Miss Islington (bot) in branch '3.9':
    bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197)
    7dfcc8e

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.8 only security fixes 3.9 only security fixes 3.10 only security fixes release-blocker topic-asyncio
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @ambv @vstinner @miss-islington