This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Non-working error handler when creating a task with assigning a variable
Type: Stage:
Components: asyncio Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, chris.jerdonek, cjrh, rshakh, yselivanov
Priority: normal Keywords:

Created on 2020-03-03 18:24 by rshakh, last changed 2022-04-11 14:59 by admin.

Messages (4)
msg363287 - (view) Author: Rustam Shakh (rshakh) Date: 2020-03-03 18:24
#This example does not work due to assigning the task to a variable

import asyncio
import logging


def handle_exception(loop, context):
    msg = context.get("exception", context["message"])
    logging.error("Caught exception: %s", msg)

async def test():
    await asyncio.sleep(1)
    raise Exception("Crash.")

def main():
    loop = asyncio.get_event_loop()
    loop.set_exception_handler(handle_exception)
    # if removed "task = " - exception handler will work.
    task = loop.create_task(test())
    try:
        loop.run_forever()
    finally:
        loop.close()


if __name__ == "__main__":
    main()
msg363290 - (view) Author: Rustam Shakh (rshakh) Date: 2020-03-03 18:32
# This line doesn`t work
task = loop.create_task(test())
    
# This line works
loop.create_task(test())
msg364274 - (view) Author: Caleb Hattingh (cjrh) * Date: 2020-03-16 01:40
Can reproduce also on 3.8. Another version that "works" (raises the exception) is

    task = loop.create_task(test())
    del task

Suggests there's something going on with reference counting or garbage collection. In the version that "doesn't work", the task exception only appears in the custom exception handler when the loop is stopped, not before. I've added a log message to show each second that passes, and the loop is stopped after 5 seconds:

    import asyncio
    import logging


    def handle_exception(loop, context):
        msg = context.get("exception", context["message"])
        logging.error("Caught exception: %s", msg)


    async def test():
        await asyncio.sleep(1)
        raise Exception("Crash.")


    def second_logger(loop, count) -> int:
        logging.warning(count)
        loop.call_later(1, lambda count=count: second_logger(loop, count + 1))


    def main():
        loop = asyncio.get_event_loop()
        loop.call_later(1, lambda: second_logger(loop, 0))
        loop.call_later(5, loop.stop)
        loop.set_exception_handler(handle_exception)
        task = loop.create_task(test())
        try:
            loop.run_forever()
        finally:
            loop.close()


    if __name__ == "__main__":
        main()

OUTPUT:

    $ py -3.8 -u bpo-issue39839.py
    WARNING:root:0
    WARNING:root:1
    WARNING:root:2
    WARNING:root:3
    WARNING:root:4
    ERROR:root:Caught exception: Crash.
msg369311 - (view) Author: Chris Jerdonek (chris.jerdonek) * (Python committer) Date: 2020-05-19 00:58
I took a look at this.

Basically, the reason the exception handler isn't firing when the task is still in scope is that the exception handler is only a handler of "last resort." You can see that the exception handler is called inside Task.__del__ here:
https://github.com/python/cpython/blob/3764069f3ba2a7e932837ae19265059339dc86e3/Lib/asyncio/tasks.py#L167-L176

The reason to do it this way is that if the Task is still in scope, there's still a chance that the caller might want to handle the exception themselves, e.g. by awaiting on the Task themselves. It's only when all references to the task go away that you know that the Task's exceptions can no longer be handled manually.

Maybe this would be worth clarifying somewhere in the Error Handling API docs:
https://docs.python.org/3/library/asyncio-eventloop.html#error-handling-api
If you agree, we can change this issue to a documentation issue.
History
Date User Action Args
2022-04-11 14:59:27adminsetgithub: 84020
2020-05-19 00:58:36chris.jerdoneksetmessages: + msg369311
2020-05-18 23:44:10chris.jerdoneksetnosy: + chris.jerdonek
2020-03-16 01:40:51cjrhsetnosy: + cjrh
messages: + msg364274
2020-03-03 18:32:53rshakhsetmessages: + msg363290
2020-03-03 18:24:30rshakhcreate