This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: [doc] asyncio event_loop fails when accessed from multiple threads
Type: behavior Stage:
Components: asyncio Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, epiphyte, iritkatriel, jnwatson, yselivanov
Priority: normal Keywords: easy

Created on 2018-09-14 15:33 by jnwatson, last changed 2022-04-11 14:59 by admin.

Messages (8)
msg325354 - (view) Author: Nic Watson (jnwatson) Date: 2018-09-14 15:33
If a signal handler callback is registered on an event loop, and the event loop has close() called on it, the close will fail.

Exception:
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/nic/.pyenv/versions/3.7.0/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/home/nic/.pyenv/versions/3.7.0/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/home/nic/tmp/signal_event_loop_bug.py", line 9, in do_loop
    loop.close()
  File "/home/nic/.pyenv/versions/3.7.0/lib/python3.7/asyncio/unix_events.py", line 58, in close
    self.remove_signal_handler(sig)
  File "/home/nic/.pyenv/versions/3.7.0/lib/python3.7/asyncio/unix_events.py", line 147, in remove_signal_handler
    signal.signal(sig, handler)
  File "/home/nic/.pyenv/versions/3.7.0/lib/python3.7/signal.py", line 47, in signal
    handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
ValueError: signal only works in main thread

Code:
import asyncio
import signal
import threading

def mysighandler():
    pass

def do_loop(loop):
    loop.close()

loop = asyncio.new_event_loop()
loop.add_signal_handler(signal.SIGINT, mysighandler)
t = threading.Thread(target=do_loop, args=(loop,))
t.start()
t.join()
msg411504 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-24 18:31
The documentation should explicitly mention that an event loop should be accessed from only one thread (it hints at this by referring to "the loop of the current thread" but I don't see this stated clearly).

In the code, it would be good if multi-threaded access failed with a clearer error.
msg411506 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-01-24 18:58
All asyncio loop methods except `loop.call_soon_threadsafe()` should be done from the same thread as been used for the loop creation.
Did you violate this rule?
msg411507 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-24 19:07
I think the script does violate the rule: 

t = threading.Thread(target=do_loop, args=(loop,))

and then do_loop closes the loop that was created in the main thread.
msg411734 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-01-26 10:41
BaseEventLoop has _check_loop() method that is closed in debug mode only.
UnixEventLoop doesn't call this method for unix-specific API.

Adding the check to add_signal_handler()/remove_signal_handler() doesn't hurt, sure. 
But it doesn't help as the script is executed in non-debug mode.

Performing a check on every call_soon() call kills the performance, that's why debug mode is required.
msg411744 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-26 12:38
Perhaps this is just a doc issue - state explicitly that a loop should be used only in one thread, and mention that this is checked in debug mode.
msg411779 - (view) Author: epiphyte (epiphyte) Date: 2022-01-26 17:06
For additional context, this was encountered when porting an application from threading to asyncio, and at the time we were still running the ioloop in its own thread as opposed to the mainthread. We no longer do that :D

Updating the documentation to clarify that the loop won't work across threads, like what we ran into at the time, would resolve this.
msg411783 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-01-26 17:31
Please feel free to propose pull request for documentation tuning.
History
Date User Action Args
2022-04-11 14:59:05adminsetgithub: 78861
2022-03-21 11:39:31iritkatrielsetkeywords: + easy
title: asyncio event_loop fails when accessed from multiple threads -> [doc] asyncio event_loop fails when accessed from multiple threads
2022-01-26 17:31:10asvetlovsetmessages: + msg411783
2022-01-26 17:06:14epiphytesetmessages: + msg411779
2022-01-26 12:38:33iritkatrielsetmessages: + msg411744
2022-01-26 10:41:20asvetlovsetmessages: + msg411734
2022-01-24 19:07:46iritkatrielsetmessages: + msg411507
2022-01-24 18:58:50asvetlovsetmessages: + msg411506
2022-01-24 18:32:49iritkatrielsettitle: asyncio event_loop close fails off main thread if signal handler registered -> asyncio event_loop fails when accessed from multiple threads
2022-01-24 18:31:53iritkatrielsetnosy: + iritkatriel

messages: + msg411504
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.6, Python 3.7
2018-09-14 15:37:34epiphytesetnosy: + epiphyte
2018-09-14 15:33:28jnwatsoncreate