This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author hniksic
Recipients asvetlov, hniksic, yselivanov
Date 2018-05-22.19:00:04
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1527015604.6.0.682650639539.issue33605@psf.upfronthosting.co.za>
In-reply-to
Content
Looking at StackOverflow's python-asyncio tag[1], it appears that it's a very common mistake for users to invoke asyncio functions or methods from a thread other than the event loop thread. In some cases this happens because the user is careless with threads and hasn't read the documentation. But in many cases what happens is that a third-party library invoked a callback in a different thread without making it transparent that that's what it's doing.

The trouble is that in many cases doing these things, e.g. calling loop.call_soon() or loop.create_task() from the wrong thread, will *appear to work*. The typical event loop is busy servicing different coroutines, so a function or task enqueued without a proper wakeup gets picked up soon enough. This is, of course, a disaster waiting to happen because it could easily lead to corruption of event loop's data structures. But users don't know that, and many of them become aware of the problem only after wondering "why does my code start working when I add a coroutine that does nothing but asyncio.sleep(0.1) in an infinite loop?" Some may never even fix their code, just assuming that asyncio takes a long time to process a new task or something like that.

I suggest that asyncio should be stricter about this error and that methods and functions that operate on the event loop, such as call_soon, call_later, create_task, ensure_future, and close, should all call _check_thread() even when not in debug mode. _check_thread() warns that it "should only be called when self._debug == True", hinting at "performance reasons", but that doesn't seem justified. threading.get_ident() is efficiently implemented in C, and comparing that integer to another cached integer is about as efficient an operation as it gets.

The added benefit would be a vast improvement of robustness of asyncio-based programs, saving many hours of debugging.


[1]
Here is an incomplete list of questions where the users stumbled on this problem, and that's only from the last three months or so:

https://stackoverflow.com/questions/49906034/python-asyncio-run-forever-and-tasks
https://stackoverflow.com/questions/49851514/python-websockets-and-gtk-confused-about-asyncio-queue
https://stackoverflow.com/questions/49533612/using-asyncio-loop-reference-in-another-file
https://stackoverflow.com/questions/49093623/strange-behaviour-when-task-added-to-empty-loop-in-different-thread
https://stackoverflow.com/questions/48836285/python-asyncio-event-wait-not-responding-to-event-set
https://stackoverflow.com/questions/48833644/how-to-wait-for-asynchronous-callback-in-the-background-i-e-not-invoked-by-us
https://stackoverflow.com/questions/48695670/running-asynchronous-code-synchronously-in-separate-thread
History
Date User Action Args
2018-05-22 19:00:05hniksicsetrecipients: + hniksic, asvetlov, yselivanov
2018-05-22 19:00:04hniksicsetmessageid: <1527015604.6.0.682650639539.issue33605@psf.upfronthosting.co.za>
2018-05-22 19:00:04hniksiclinkissue33605 messages
2018-05-22 19:00:04hniksiccreate