classification
Title: Exceptions raised by EventLoop.call_soon_threadsafe
Type: Stage: resolved
Components: asyncio Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Ben.Darnell, asvetlov, cmeyer, lukasz.langa, mikeshardmind, miss-islington, vstinner, yselivanov
Priority: release blocker Keywords: patch

Created on 2020-02-16 16:41 by Ben.Darnell, last changed 2020-09-12 07:11 by miss-islington. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 22197 merged vstinner, 2020-09-11 10:40
PR 22215 merged miss-islington, 2020-09-12 06:50
PR 22216 merged miss-islington, 2020-09-12 06:51
Messages (10)
msg362078 - (view) Author: Ben Darnell (Ben.Darnell) * Date: 2020-02-16 16:41
Proactor and selector event loops behave differently when call_soon_threadsafe races with a concurrent call to loop.close(). In a selector event loop, call_soon_threadsafe will either succeed or raise a RuntimeError("Event loop is closed"). In a proactor event loop, it could raise this RuntimeError, but it can also raise an AttributeError due to an unguarded access to self._csock. 

https://github.com/python/cpython/blob/1ed61617a4a6632905ad6a0b440cd2cafb8b6414/Lib/asyncio/proactor_events.py#L785-L787

Comments in BaseSelectorEventLoop._write_to_self indicate that this is deliberate, so the `csock is not None` check here should probably be copied to the proactor event loop version.

https://github.com/python/cpython/blob/1ed61617a4a6632905ad6a0b440cd2cafb8b6414/Lib/asyncio/selector_events.py#L129-L136


I'd also accept an answer that the exact behavior of this race is undefined and it's up to the application to either arrange for all calls to call_soon_threadsafe to stop before closing the loop. However, I've had users of Tornado argue that they use the equivalent of call_soon_threadsafe in contexts where this coordination would be difficult, and I've decided that tornado's version of this method would never raise, even if there is a concurrent close. So if asyncio declines to specify which exceptions are allowed in this case, tornado will need to add a blanket `except Exception:` around calls to call_soon_threadsafe.
msg366793 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2020-04-19 17:14
Good catch. We should fix this for Python 3.8.3.
msg366839 - (view) Author: Chris Meyer (cmeyer) * Date: 2020-04-20 15:06
Is this related to bpo-39010 too?
msg366840 - (view) Author: Ben Darnell (Ben.Darnell) * Date: 2020-04-20 15:10
No, this is unrelated to bpo-39010.
msg368791 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2020-05-13 17:25
This sadly missed 3.8.3 but I want this addressed for 3.8.4.
msg373594 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2020-07-13 11:36
We have trouble finding a Windows expert with available time to address this :/ This is missing 3.8.4 as well.
msg376784 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-12 06:50
New changeset 1b0f0e3d7d03155da1cf9769a847874d559e57e3 by Victor Stinner in branch 'master':
bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197)
https://github.com/python/cpython/commit/1b0f0e3d7d03155da1cf9769a847874d559e57e3
msg376785 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-12 06:53
Thanks for the bug report Ben Darnell, it's now fixed.
msg376790 - (view) Author: miss-islington (miss-islington) Date: 2020-09-12 07:10
New changeset 530d1105ed7b0aa5ef76e3116918fe39bc6a4823 by Miss Islington (bot) in branch '3.8':
bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197)
https://github.com/python/cpython/commit/530d1105ed7b0aa5ef76e3116918fe39bc6a4823
msg376791 - (view) Author: miss-islington (miss-islington) Date: 2020-09-12 07:11
New changeset 7dfcc8e0795cce6a8cb42ae2a7f159a38da6b38a by Miss Islington (bot) in branch '3.9':
bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197)
https://github.com/python/cpython/commit/7dfcc8e0795cce6a8cb42ae2a7f159a38da6b38a
History
Date User Action Args
2020-09-12 07:11:42miss-islingtonsetmessages: + msg376791
2020-09-12 07:10:01miss-islingtonsetmessages: + msg376790
2020-09-12 06:53:41vstinnersetstatus: open -> closed
versions: + Python 3.9, Python 3.10
messages: + msg376785

resolution: fixed
stage: patch review -> resolved
2020-09-12 06:51:02miss-islingtonsetpull_requests: + pull_request21271
2020-09-12 06:50:54miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request21270
2020-09-12 06:50:26vstinnersetmessages: + msg376784
2020-09-11 10:40:49vstinnersetkeywords: + patch
nosy: + vstinner

pull_requests: + pull_request21256
stage: patch review
2020-07-13 11:36:00lukasz.langasetmessages: + msg373594
2020-05-13 17:25:56lukasz.langasetmessages: + msg368791
2020-04-20 15:10:12Ben.Darnellsetmessages: + msg366840
2020-04-20 15:06:04cmeyersetnosy: + cmeyer
messages: + msg366839
2020-04-19 17:14:51lukasz.langasetpriority: normal -> release blocker
nosy: + lukasz.langa
messages: + msg366793

2020-02-18 03:24:38mikeshardmindsetnosy: + mikeshardmind
2020-02-16 16:41:58Ben.Darnellcreate