How can I yield notifications and return a result from a function? (Python)

Question

I have to create a function that does some hard work in the internal calls. This function needs to be a generator because I'm using Server-Sent Events. So that, I want that this function notifies the progress of the calculations by using a "yield". After that, this function has to pass the result to the parent function in order to continue with other calculations.

I would like something like this:

def hardWork():
    for i in range(N):
        # hard work
        yield 'Work done: ' + str(i)

    # Here is the problem: I can't return a result if I use a yield
    return result             

def generator():
    # do some calculations
    result = hardWork()
    # do other calculations with this result
    yield finalResult

I found a solution that consists on yields a dictionary that tells if the function has finished or not, but the code to do this is pretty dirty.

Is there another solution?

Thank you!

EDIT

I thought something like:

def innerFunction(gen):
    calc = 1

    for iteration in range(10):
        for i in range(50000):
            calc *= random.randint(0, 10)
        gen.send(iteration)

    yield calc


def calcFunction(gen):
    gen2 = innerFunction(gen)
    r = next(gen2)

    gen.send("END: " + str(r + 1))
    gen.send(None)


def notifier():
    while True:
        x = yield
        if x is None:
            return
        yield "Iteration " + x


def generator():
    noti = notifier()
    calcFunction(noti)
    yield from noti


for g in generator():
    print(g)

But I'm receiving this error:

TypeError: can't send non-None value to a just-started generator

see this question: https://stackoverflow.com/questions/16780002/return-in-generator-together-with-yield-in-python-3-3 — matejcik, Sep 06 '18 at 13:19
I think you should really read about async functions as this is very close to what you are doing — Olivier Melançon, Sep 06 '18 at 13:34

Olivier Melançon · Accepted Answer · 2018-09-06T16:59:30.187

Prior to Python3.5: generators

This solution also works for more recent Python versions, although async def, new in Python3.5, seems more fit for your usecase. See next section.

The values yielded by a generator are obtained by iterating or using next. The value returned at the end is stored in the value attribute of the StopIteration exception that indicates the end of the generator. Fortunately, it is not too hard to recover.

def hardWork():
    output = []

    for i in range(10):
        # hard work
        yield 'Doing ' + str(i)
        output.append(i ** 2)

    return output

def generator():
    # do some calculations
    work = hardWork()

    while True:
        try:
            print(next(work))
        except StopIteration as e:
            result = e.value
            break

    yield result

Example

foo = generator()
print(next(foo))

Output

Doing 0
Doing 1
Doing 2
Doing 3
Doing 4
Doing 5
Doing 6
Doing 7
Doing 8
Doing 9
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Python3.5+: `async def`

If you are running Python3.5+, what you are attempting seems perfectly fit for an event loop using awaitable functions.

import asyncio

async def hardWork():
    output = []

    for i in range(10):
        # do hard work
        print('Doing ', i)
        output.append(i**2)

        # Break point to allow the event loop to do other stuff on the side
        await asyncio.sleep(0)

    return output

async def main():
    result = await asyncio.wait_for(hardWork(), timeout=None)
    print(result)

loop = asyncio.get_event_loop()

loop.run_until_complete(main())

Output

Doing  0
Doing  1
Doing  2
Doing  3
Doing  4
Doing  5
Doing  6
Doing  7
Doing  8
Doing  9
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

score 0 · Answer 2 · answered Sep 06 '18 at 13:58

My suggestion would be to embed your functionality in a class.

def Worker:

    def hardWork(self):
        self.Finished = False
        for i in range(10):
            yield 'Work done: ' + str(i)
        self.Finished = True
        self.Result = 'result'

    def generator(self):
        while (not self.Finished):
            print(next(self.hardWork()))
        return self.Result

This will have the functionality you want without having to worry about programming your logic around exception throw-catch logic.

krubo · Answer 3 · 2019-10-26T14:57:20.537

Python 3.3 introduces yield from for generator delegation, which is pretty much exactly what you're asking for. yield from allows the main generator to delegate functionality to another function which also contains yield statements, like this:

def hardWork():
    for i in range(N):
        # hard work
        yield 'Work done so far: ' + str(i)

    # With this construction, you can still return a result after the `yield`s
    return result

def generator():
    # here's the new construction that makes it all work:
    result = yield from hardWork()

    # do other calculations with this result
    yield finalResult

How can I yield notifications and return a result from a function? (Python)

EDIT

3 Answers3

Prior to Python3.5: generators

Example

Output

Python3.5+: async def

Output

Python3.5+: `async def`