python asyncio exceptions raised from loop.create_task()

Question

I want my code to use python logging to log exceptions. In my usual code using await, exceptions are raised normally, so:

try: await code_that_can_raise() except Exception as e: logger.exception("Exception happended")

Works fine.

However, when using loop.create_task(coro())

I'm not sure how can I catch the exception here.
Wrapping the create_task() call obviously won't work. What is the best solution to log every exception in the code?

user4815162342 · Answer 1 · 2019-10-29T08:03:14.520

7

What is the best solution to log every exception in the code?

If you control the invocation of create_task, but don't control the code in the coro(), then you can write a logging wrapper:

async def log_exceptions(awaitable):
    try:
        return await awaitable
    except Exception:
        logger.exception("Unhandled exception")

then you can call loop.create_task(log_exceptions(coro())).

If you can't or don't want to wrap every create_task, you can call loop.set_exception_handler, setting the exception to your own function that will log the exception as you see fit.

edited Oct 29 '19 at 08:03

answered Apr 16 '19 at 12:43

user4815162342

124,516
15
228
298

2

It's [better](https://stackoverflow.com/a/14797463/1113207) to use `except Exception:` instead of `except:` to avoid suppressing `KeyboardInterrupt` and similar non-errors. – Mikhail Gerasimov Apr 16 '19 at 13:06
Any general solution that is based on the event loop? rather than wrapping every create_task call. Since as I understand, the exception is eventually raised up to the event loop. – user3599803 Apr 16 '19 at 14:04
@user3599803 You can use `set_exception_handler` for that purpose; see the editer answer. – user4815162342 Apr 16 '19 at 14:09
how do you await in a function that is not async? – nurettin Oct 29 '19 at 07:38
@nurettin Good catch, I've now added the missing `async`. – user4815162342 Oct 29 '19 at 08:04

score 4 · Answer 2 · answered Apr 16 '19 at 14:12

Just so that it has been mentioned: asyncio.Task objects have the methods result and exception.
result:

[...] if the coroutine raised an exception, that exception is re-raised [...]

exception:

[...] If the wrapped coroutine raised an exception that exception is returned [...]

Given a simple setup (in Python 3.7 syntax):

import asyncio
tasks =[]

async def bad_test():
    raise ValueError

async def good_test():
    return

async def main():
    tasks.append(asyncio.create_task(bad_test()))
    tasks.append(asyncio.create_task(good_test()))

asyncio.run(main())

Using result, one could do:

for t in tasks:
    try:
        f = t.result()
    except ValueError as e:
        logger.exception("we're all doomed")

Or, using exception:

for t in tasks:
    if isinstance(t.exception(), Exception):
        logger.exception("apocalypse now")

However, both methods require the Task to be done, otherwise:

If the Task has been cancelled, this method raises a CancelledError exception.

(result): If the Task’s result isn’t yet available, this method raises a InvalidStateError exception.

(exception): If the Task isn’t done yet, this method raises an InvalidStateError exception.

So, unlike the proposal in the other answer, the logging will not happen when the exceptions raise in the tasks, but rather when the tasks are evaluated after they completed.

score 0 · Answer 3 · answered Nov 30 '21 at 11:42

Expanding on @user4815162342's solution, I created a wrapper around log_exceptions to avoid having to nest every coroutine inside 2 functions:

import asyncio
from typing import Awaitable

def create_task_log_exception(awaitable: Awaitable) -> asyncio.Task:
    async def _log_exception(awaitable):
        try:
            return await awaitable
        except Exception as e:
            logger.exception(e)
    return asyncio.create_task(_log_exception(awaitable))

Usage:

create_task_log_exception(coroutine())

python asyncio exceptions raised from loop.create_task()

3 Answers3