classification
Title: asyncio run_forever blocks indefinitely
Type: behavior Stage: resolved
Components: asyncio, macOS Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, dantimofte, ned.deily, ronaldoussoren, yselivanov
Priority: normal Keywords: patch

Created on 2019-04-14 05:57 by dantimofte, last changed 2019-04-17 13:07 by asvetlov. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 12845 closed dantimofte, 2019-04-15 15:41
Messages (6)
msg340182 - (view) Author: Dan Timofte (dantimofte) * Date: 2019-04-14 05:57
after starting run_forever if all scheduled tasks are consumed run_once will issue a KqueueSelector.select(None) which will block indefinitely :
https://www.freebsd.org/cgi/man.cgi?query=select&sektion=2&apropos=0&manpath=FreeBSD+12.0-RELEASE+and+Ports#DESCRIPTION

after this new tasks are not being processed, trying to stop event loop with stop() is not working.

this blocks immediatly : 
import asyncio
import sys
import signal


def cb_signal_handler(signum, frame):
    asyncio.get_event_loop().stop()


def main():
    signal.signal(signal.SIGINT, cb_signal_handler)

    # asyncio.get_event_loop().create_task(asyncio.sleep(1))
    asyncio.get_event_loop().run_forever()

main()

With asyncio.sleep uncomment it will block after 4 cycles.
msg340204 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-04-14 12:27
Callin `self._write_to_self()` from `loop.stop()` should fix your problem.

Would you provide a patch?
msg340216 - (view) Author: Dan Timofte (dantimofte) * Date: 2019-04-14 15:45
i will provide a patch, i'll make a pull request next week.

a call to self._write_to_self() should also be added to create_task() before it returns . i'll make the correction for this as well.
msg340332 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-04-16 10:30
Not sure about `create_task()`.
Usually you create tasks from async code, where the `_write_to_self()` call is not needed.

Handling writing to self-pipe is not free, starting very many tasks at once can hit performance.

Stopping the loop is another beast, we can perform relative slow operations in such calls.

Thinking more about the issue I'm inclining to reject my initial proposal.

If you want to stop a loop from signal handler you should make `loop.call_soon_threadsafe(loop.stop)` call because `loop.stop()` is not thread-safe operation by definition.

Threadsafe call solves your problem, isn't it?
msg340392 - (view) Author: Dan Timofte (dantimofte) * Date: 2019-04-17 12:27
`loop.call_soon_threadsafe(loop.stop)` solves the problem because it has the write_to_self there. I can use that or call loop._write_to_self() myself before calling loop.stop(). 

In my code i'm stoping the loop from the exception_handler not signal. The code was a small example i thought of that reproduces the described behaviour.

You can close the issue if it's ok for you.
msg340393 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-04-17 13:07
Technically signal handlers are called from main thread, while loop can be executed in another one.
*In general* `call_soon_threadsafe()` is the correct solution.
Also, it works fine just now with Python 3.5+

Let's close as won't fix
History
Date User Action Args
2019-04-17 13:07:50asvetlovsetstatus: open -> closed
resolution: wont fix
stage: patch review -> resolved
2019-04-17 13:07:37asvetlovsetmessages: + msg340393
2019-04-17 12:27:00dantimoftesetmessages: + msg340392
2019-04-16 10:30:27asvetlovsetmessages: + msg340332
2019-04-15 15:41:46dantimoftesetkeywords: + patch
stage: patch review
pull_requests: + pull_request12770
2019-04-14 15:45:55dantimoftesetmessages: + msg340216
2019-04-14 12:27:20asvetlovsetversions: + Python 3.8
2019-04-14 12:27:10asvetlovsetmessages: + msg340204
2019-04-14 06:08:53dantimoftesettype: behavior
2019-04-14 05:57:31dantimoftecreate