Coroutines in Magic Methods?

Question

let's say, I've got a magic Method in Python 3.6 like:

def __str__(self):
    return 'a {self.color} car'.format(self=self)

Now I want to add a Coroutine into it, something like:

async def __str__(self):
    await self.my_coro(args)
    return 'a {self.color} car'.format(self=self)

As far as I know, it is necessary to add async before def, but this seems not to work with magic Methods. Is there an (easy) workaround or totally impossible?

This would require that all consumers (callers) of this API will handle it as an async method, which is not defined in its behaviour. In other words, nobody expects `__str__` to be async, so nobody will `await` it, so this won't work. This isn't specific to magic methods really, you'll have this every time you're implementing an already defined interface. — deceze, Jan 30 '20 at 10:38
What are you trying to accomplish, specifically? In the example with `__str__` it doesn't seem necessary to use `await` in order to return a formatted string. — mkrieger1, Jan 30 '20 at 10:39
Not to mention that a method like `__str__` shouldn't be so complex/expensive as to include an async call. — deceze, Jan 30 '20 at 10:39
@mkrieger1 I wanted to replace a sync function into a coroutine in a project. But the rat tail is realy long (because now I have to make await statements everywhere the old function/new coroutine was/is used etc.), so I try to replace every sync method with an async method, but my dead end is a `__exit__` Method. — zonk, Jan 30 '20 at 10:45
See [PEP 492](https://www.python.org/dev/peps/pep-0492/#asynchronous-context-managers-and-async-with). — mkrieger1, Jan 30 '20 at 10:48
@zonk Are you asking for special methods *in general* or context managers *in specific*? — MisterMiyagi, Jan 30 '20 at 10:50
@mkrieger1 tbh I have no clue about Asynchrounous context managers. I'll have a look on it, if this could be a usefull hint for the problem. — zonk, Jan 30 '20 at 11:03
@MisterMiyagi In my case it is a special method and I believe I will hit other special methods as well, so I would say _"special methods in general"_ — zonk, Jan 30 '20 at 11:03
`__enter__` and `__exit__` are used to implement context managers. If you feel the need to use `await` in them, then you need an asynchronous context manager, which you can implement either by replacing those methods with `__aenter__` and `__aexit__` or by using one of the solutions from the question I've linked. — mkrieger1, Jan 30 '20 at 11:05

MisterMiyagi · Accepted Answer · 2020-01-30T11:42:13.683

TLDR: There is no workaround as Python expects a specific type for many special methods. However, some special methods have async variants.

An async def function is fundamentally a different kind of function. Similar to how a generator function evaluates to a generator, a coroutine function evaluates to an awaitable.

You can define special methods to be async, but that affects their return type. In the case of __str__, you get an Awatable[str] instead of a bare str. Since Python requires a str here, this results in an error.

>>> class AStr:
...     async def __str__(self):
...         return 'Hello Word'
>>> str(AStr())

TypeError: __str__ returned non-string (type coroutine)

This affects all special methods that are directly interpreted by Python: These include __str__, __repr__, __bool__, __len__, __iter__, __enter__ and some others. Usually, if a special method relates to some internally used functionality (e.g. str for direct display) or statement (e.g. for needing an iterator) it cannot be async.

Some special methods are not directly interpreted by Python. Examples include arithmetic operators (__add__, __sub__, ...), comparisons (__lt__, __eq__, ...) and lookup (__get__, __getattribute__, ...). Their return type can be any object, including awaitables.

You can define such special methods via async def. This affects their return type, but only requires client code to await them. For example, you can define + to be used as await (a + b).

>>> def AWAIT(awaitable):
...     """Basic event loop to allow synchronous ``await``"""
...     coro = awaitable.__await__()
...     try:
...         while True:
...             coro.send(None)
...     except StopIteration as e:
...         return e.args[0] if e.args else None
...
>>> class APlus:
...     def __init__(self, value):
...         self.value = value
...     async def __add__(self, other):
...         return self.value + other
...
>>> async def add(start, *values):
...     total = start
...     for avalue in map(APlus, values):
...         total = await (avalue + total)
...     return total
...
>>> AWAIT(add(5, 10, 42, 23))
80

A few special methods exist for the await machinery and are expected to return an awaitable. This includes __aenter__, __aexit__, and __anext__. Notably, __await__ must return an iterator, not an awaitable.

You can (and in most cases should) define these methods as async def. If you need asynchronous capabilities in the corresponding sync special method, use the corresponding async special method with async def. For example, you can define an asynchronous context manager.

>>> class AContext:
...     async def __aenter__(self):
...         print('async enter')
...     async def __aexit__(self, exc_type, exc_val, exc_tb):
...         print('async exit')
...
>>> async def scoped(message):
...     async with AContext():
...         print(message)
...
>>> AWAIT(scoped("Hello World"))
async enter
Hello World
async exit

Coroutines in Magic Methods?

1 Answers1