How can I wrap a synchronous function in an async coroutine?

python python-3.x asynchronous python-asyncio aiohttp
Zac Delventhal picture Zac Delventhal · Apr 5, 2017 · Viewed 19.1k times · Source

I'm using aiohttp to build an API server that sends TCP requests off to a seperate server. The module that sends the TCP requests is synchronous and a black box for my purposes. So my problem is that these requests are blocking the entire API. I need a way to wrap the module requests in an asynchronous coroutine that won't block the rest of the API.

So, just using sleep as a simple example, is there any way to somehow wrap time-consuming synchronous code in a non-blocking coroutine, something like this:

async def sleep_async(delay):
    # After calling sleep, loop should be released until sleep is done
    yield sleep(delay)
    return 'I slept asynchronously'

Answer

Zac Delventhal picture Zac Delventhal · Apr 6, 2017

Eventually I found an answer in this thread. The method I was looking for is run_in_executor. This allows a synchronous function to be run asynchronously without blocking an event loop.

In the sleep example I posted above, it might look like this:

import asyncio
from time import sleep

async def sleep_async(loop, delay):
    # None uses the default executor (ThreadPoolExecutor)
    await loop.run_in_executor(None, sleep, delay)
    return 'I slept asynchronously'

Also see the following answer -> How do we call a normal function where a coroutine is expected?

Related questions

Multiple loops with asyncio

Is it possible to have multiple loops with asyncio? If the response is yes how can I do that? My use case is: * I extract urls from a list of websites in async * For each "sub url list", I would …

Read More
TypeError: An asyncio.Future, a coroutine or an awaitable is required

I'm trying to make an asynchronous web scraper using beautifulsoup and aiohttp.This is my initial code to start things.I'm getting a [TypeError: An asyncio.Future, a coroutine or an awaitable is required] and having a hard time figuring …

Read More
how to add a coroutine to a running asyncio loop?

How can one add a new coroutine to a running asyncio loop? Ie. one that is already executing a set of coroutines. I guess as a workaround one could wait for existing coroutines to complete and then initialize a new …

Read More