This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: asyncio.run does not allow for graceful shutdown of main task
Type: behavior Stage: resolved
Components: asyncio Versions:
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: andreash, asvetlov, yselivanov
Priority: normal Keywords:

Created on 2021-07-31 12:44 by andreash, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (2)
msg398638 - (view) Author: Andreas H. (andreash) * Date: 2021-07-31 12:44
The issue is that the main task (which was supplied to asyncio.run) has no chance to clean up its "own" sub-tasks and handle 
possible exceptions that occur during the sub-task clean up. It prevents a graceful shutdown.

There is no way to prevent the current printing of the "unhandled" exeption, even though the sub-task exception was catched by the main task. (See example below)


-- Current behavior --

When asyncio.run() receives an (unhanded) exception, all tasks are cancelled simultaneously. 
	
If any task generates an exception during its clean-up phase this is printed to the log, even though this exception is handled by the main task.
	

-- Expected behavior --
	
asyncio.run() should first cancel the main task, wait for it to complete its shutdown (and possible cancel its own sub-tasks, with exception catching), and *afterwards* cancel the remaining tasks.
	
	
-- Example Code --

For instance realize a graceful shutdown of a webserver when SIGTERM signal handler raises a SystemExit exception.




import os
import asyncio
import logging


async def main():

    logging.basicConfig(level=logging.INFO)

    async def sub_task():
        logging.info('sub_task: enter')    
        try:    
            while True:
                await asyncio.sleep(1)
                logging.info('some_task: action')
        finally:
            logging.info('sub_task: cleanup')    
            await asyncio.sleep(3)
            logging.info('sub_task: cleanup generates exception')    
            raise ValueError()
            logging.info('sub_task: cleanup end')    
    
    task = asyncio.create_task(sub_task())
                         
    try:
        while True:
            await asyncio.sleep(1)
    except Exception as e:
        logging.info(f"Main: exception {repr(e)} received: something went wrong: cancelling sub-task")
        task.cancel()
    finally:
        logging.info("Main: cleanup")
        try:
            await task
        except Exception as e:
            logging.info(f"Main: catched exception {repr(e)} from await sub_task")

try:
    asyncio.run( main() )
except KeyboardInterrupt:
    pass

-- Script Output with Ctrl+C manually generating an KeyboardInterrupt exception --


INFO:root:sub_task: enter
INFO:root:some_task: action    
        <--- CtrlC pressed here
INFO:root:Main: exception CancelledError() received: something went wrong: cancelling sub-task
INFO:root:Main: cleanup
INFO:root:sub_task: cleanup
INFO:root:sub_task: cleanup generates exception
INFO:root:Main: catched exception ValueError() from await sub_task
ERROR:asyncio:unhandled exception during asyncio.run() shutdown
task: <Task finished coro=<main.<locals>.sub_task() done, defined at D:\Benutzer\projekte\iep\apps\data_player\_signals_test\test.py:10> exception=ValueError()>
Traceback (most recent call last):
  File "C:\Users\z0013xar\AppData\Local\Continuum\anaconda3\lib\asyncio\runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "C:\Users\z0013xar\AppData\Local\Continuum\anaconda3\lib\asyncio\base_events.py", line 574, in run_until_complete
    self.run_forever()
  File "C:\Users\z0013xar\AppData\Local\Continuum\anaconda3\lib\asyncio\base_events.py", line 541, in run_forever
    self._run_once()
  File "C:\Users\z0013xar\AppData\Local\Continuum\anaconda3\lib\asyncio\base_events.py", line 1750, in _run_once
    event_list = self._selector.select(timeout)
  File "C:\Users\z0013xar\AppData\Local\Continuum\anaconda3\lib\selectors.py", line 323, in select
    r, w, _ = self._select(self._readers, self._writers, [], timeout)
  File "C:\Users\z0013xar\AppData\Local\Continuum\anaconda3\lib\selectors.py", line 314, in _select
    r, w, x = select.select(r, w, w, timeout)
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Benutzer\projekte\iep\apps\data_player\_signals_test\test.py", line 14, in sub_task
    await asyncio.sleep(1)
  File "C:\Users\z0013xar\AppData\Local\Continuum\anaconda3\lib\asyncio\tasks.py", line 595, in sleep
    return await future
concurrent.futures._base.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Benutzer\projekte\iep\apps\data_player\_signals_test\test.py", line 34, in main
    await task
  File "D:\Benutzer\projekte\iep\apps\data_player\_signals_test\test.py", line 20, in sub_task
    raise ValueError()
ValueError

-- Expected Output --

Same as above but without

  "ERROR:asyncio:unhandled exception during asyncio.run() shutdown"

and following traceback
msg414994 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2022-03-12 15:43
Please use TaskGroup from Python 3.11 for structural concurrency.
History
Date User Action Args
2022-04-11 14:59:48adminsetgithub: 88958
2022-03-12 15:43:41asvetlovsetstatus: open -> closed
resolution: out of date
messages: + msg414994

stage: resolved
2021-07-31 12:44:30andreashcreate