Removing async pollution from Python

Question

How do I remove the async-everywhere insanity in a program like this?

import asyncio


async def async_coro():
    await asyncio.sleep(1)


async def sync_func_1():
    # This is blocking and synchronous
    await async_coro()


async def sync_func_2():
    # This is blocking and synchronous
    await sync_func_1()


if __name__ == "__main__":
    # Async pollution goes all the way to __main__
    asyncio.run(sync_func_2())

I need to have 3 async markers and asyncio.run at the top level just to call one async function. I assume I'm doing something wrong - how can I clean up this code to make it use async less?

FWIW, I'm interested mostly because I'm writing an API using asyncio and I don't want my users to have to think too much about whether their functions need to be def or async def depending on whether they're using a async part of the API or not.

[Async just does this.](https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/) — user2357112, Oct 28 '19 at 01:28
`async.sleep()` does not sleep in the classical sense. It schedules a continuation (in this case for an implicit `return None`) and hands back control to the loop. — Klaus D., Oct 28 '19 at 01:38
I mean, call it whatever you want, but the program takes 1 second to run. It definitely does not "do nothing" — Thomas Johnson, Oct 28 '19 at 01:41
It takes 1 second to run because the main thread is open during the sleep. What @KlausD. is trying to tell you is that `async.sleep()` will sleep **only the function that called it**. — felipe, Oct 28 '19 at 01:50
In other words, if you call two `async` functions: `function1(); function2()` and `function1()` had a `async.sleep()` statement, `async` would switch its process from `function1()` to `function2()` on the sleep statement. Your confusion is coming from a lack of understanding on how asynchronous code work, which is why @user2357112 referred you to a link. — felipe, Oct 28 '19 at 01:52
Maybe I'm misunderstanding, but I don't think the details of the sleep call matter unless it's relevant to removing the `async`s. It's just a placeholder for more complicated async logic. The link given by @user2357112 is basically the same complaint as mine, and essentially says it's not possible to remove the asyncs. — Thomas Johnson, Oct 28 '19 at 01:54
To answer your question directly, you have to use the `async def function()` to define an asynchronous function in Python. You do not, however, have to call your functions `async_function_name()` or `sync_function_name`. A synchronous function is one that is simply defined as `def function()`, where an asynchronous one is called `async def function()`. — felipe, Oct 28 '19 at 01:56
Therefore, technically, your `sync_func_2` and `sync_func_1` are asynchronous functions (deleted comment before because accidentally wrote "aren't" asynchronous functions). — felipe, Oct 28 '19 at 02:20
@ThomasJohnson consider reading [this answer](https://stackoverflow.com/a/57734557/1113207) about why `asyncio` forces you to use async/await and [this answer](https://stackoverflow.com/a/33399896/1113207) about async programming in general. I think it may help you solve the issue. — Mikhail Gerasimov, Oct 28 '19 at 09:13

score 1 · Answer 1 · answered Oct 28 '19 at 03:35

1

After some research, one answer is to manually manage the event loop:

import asyncio


async def async_coro():
    await asyncio.sleep(1)


def sync_func_1():
    # This is blocking and synchronous
    loop = asyncio.get_event_loop()
    coro = async_coro()
    loop.run_until_complete(coro)


def sync_func_2():
    # This is blocking and synchronous
    sync_func_1()


if __name__ == "__main__":
    # No more async pollution
    sync_func_2()

answered Oct 28 '19 at 03:35

Thomas Johnson

10,776
18
60
98

That won't work if one sync function implemented in this fashion at any point needs to call another such function - you'll get an "event loop is running" RuntimeError. Also, setup and teardown of an event loop has a cost which is non-negligible if done behind the scenes - an empty `asyncio.run` takes around 0.15ms on my machine. – user4815162342 Oct 28 '19 at 07:15
Added an alternative answer that avoids this issue. – user4815162342 Oct 28 '19 at 13:25

user4815162342 · Answer 2 · 2019-10-28T19:36:44.107

If you must do that, I would recommend an approach like this:

import asyncio, threading

async def async_coro():
    await asyncio.sleep(1)

_loop = asyncio.new_event_loop()
threading.Thread(target=_loop.run_forever, daemon=True).start()

def sync_func_1():
    # This is blocking and synchronous
    return asyncio.run_coroutine_threadsafe(async_coro(), _loop).result()

def sync_func_2():
    # This is blocking and synchronous
    sync_func_1()

if __name__ == "__main__":
    sync_func_2()

The advantage of this approach compared to one where sync functions run the event loop is that it supports nesting of sync functions. It also only runs a single event loop, so that if the underlying library wants to set up e.g. a background task for monitoring or such, it will work continuously rather than being spawned each time anew.

Removing async pollution from Python

2 Answers2