Lazy iterators (generators) with asyncio

Question

I have a blocking, non-async code like this:

def f():
    def inner():
        while True:
            yield read()
    return inner()

With this code the caller can choose when to stop the function to generate data. How to change this to async? This solution doesn't work:

async def f():
    async def inner():
        while True:
            yield await coroutine_read()
    return inner()

... because yield can't be used in async def functions. If i remove the async from the inner() signature, I can't use await anymore.

You don't need `yield read()` when using asyncio, because `await` will do exactly this behind the scenes. This doesn't answer the question of course. — Sergei Lebedev, May 17 '16 at 15:25
PEP-0492 [doesn't cover](https://www.python.org/dev/peps/pep-0492/#coroutine-generators) coroutine-generators (which is what you want), so, since the PEP was only implemented in 3.5, I guess the answer is "there's no way to do this". — Sergei Lebedev, May 17 '16 at 15:29
Attempt to implement way to yield inside async functions: http://stackoverflow.com/a/37572657/1113207 — Mikhail Gerasimov, Jun 01 '16 at 15:21
Possible duplicate of [How to use 'yield' inside async function?](http://stackoverflow.com/questions/37549846/how-to-use-yield-inside-async-function) — rominf, Sep 18 '16 at 11:57

Mikhail Gerasimov · Accepted Answer · 2019-04-07T08:52:45.483

Upd:

Starting with Python 3.6 we have asynchronous generators and able to use yield directly inside coroutines.

As noted above, you can't use yield inside async funcs. If you want to create coroutine-generator you have to do it manually, using __aiter__ and __anext__ magic methods:

import asyncio


# `coroutine_read()` generates some data:
i = 0
async def coroutine_read():
    global i
    i += 1
    await asyncio.sleep(i)
    return i


# `f()` is asynchronous iterator.
# Since we don't raise `StopAsyncIteration` 
# it works "like" `while True`, until we manually break.
class f:
    async def __aiter__(self):
        return self

    async def __anext__(self):
        return await coroutine_read()


# Use f() as asynchronous iterator with `async for`:
async def main():
    async for i in f():
        print(i)
        if i >= 3:
            break


if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

Output:

1
2
3
[Finished in 6.2s]

You may also like to see other post, where StopAsyncIteration uses.

Lazy iterators (generators) with asyncio

1 Answers1

Linked