calling sync functions from async function

chris picture chris · Feb 14, 2019 · Viewed 10.2k times · Source

I am in the process of trying to port a flask app to quart to utilise asyncio. I don't think my current approach is working, as my entire function chain is written without async in mind - consider the following:

def long_running_task(task):
    result = some_synchronous_function(task)
    return result

@app.route('/<task>', methods=['GET'])
async def do_task(task):
    ok = await long_running_task(task)
    if ok:
        return (ok.result)
    else:
        return ('Something went wrong')

If long_running_task and its whole chain of function calls are not declared as async, am I actually getting any benefit from my route being declared as async?

Answer

J. Taylor picture J. Taylor · Feb 14, 2019

To run a blocking synchronous function from asyncio, without blocking the main event loop, you can use loop.run_in_executor() to run the blocking function in a ThreadPoolExecutor or ProcessPoolExecutor` (i.e. in its own thread or process).

From within the async function you want to call it from:

loop = asyncio.get_event_loop()

result = await loop.run_in_executor(None, long_running_task, task)

The first argument None is to tell it to use the default executor for the loop. Obviously do_task() will still have to wait for result to complete, but while it is waiting, other async tasks will be able to run in event-loop.