async - sync - async calls in one python event loop

dzmitry picture dzmitry · Jul 20, 2017 · Viewed 8.9k times · Source

Let's say I have a class which uses asyncio loop internally and doesn't have async interface:

class Fetcher:
    _loop = None
    def get_result(...):
        """
        After 3 nested sync calls async tasks are finally called with *run_until_complete*
        """
        ...

I use all advantages of asyncio internally and don't have to care about it in the outer code.

But then I want to call 3 Fetcher instances in one event loop. If I had async def interface there would be no problem: asyncio.gather could help me. Is there really no other way to do it without supporting both interfaces? Come on! It makes you change all your project because of one asyncio usage. Tell me this is not true.

Answer

Mikhail Gerasimov picture Mikhail Gerasimov · Jul 20, 2017

Come on! It makes you change all your project because of one asyncio usage. Tell me this is not true.

It's true

Whole idea of using await keyword is to execute concurrent jobs in one event loop from different places of the code (which you can't do with regular function code).

asyncio - is not some utility, but whole style of writing asynchronous programs.


On the other hand Python is very flexible, so you can still try to hide using of asyncio. If you really want to get sync result of 3 Fetcher instances, you can for example do something like this:

import asyncio


def sync_exec(coro):
    loop = asyncio.get_event_loop()
    return loop.run_until_complete(coro)


class Fetcher:
    async def async_get_result(self):
        # async interface:
        async def async_job():
            await asyncio.sleep(1)
            return id(self)
        return (await async_job())

    def get_result(self):
        # sync interface:
        return sync_exec(self.async_get_result())

    @classmethod
    def get_results(cls, *fetchers):
        # sync interface multiple:
        return sync_exec(
            asyncio.gather(*[fetcher.async_get_result() for fetcher in fetchers])
        )        



# single sync get_result:
f1 = Fetcher()
print('Result: ', f1.get_result())


# multiple sync get_result:
f2 = Fetcher()
f3 = Fetcher()
print('Results: ', Fetcher.get_results(f1, f2, f3))

Output:

Result:  2504097887120
Results:  [2504097887120, 2504104854416, 2504104854136]

But, again, you'll really regret someday if you continue to write code this way, believe me. If you want to get full advantage of asynchronous programming - use coroutines and await explicitly.