98

What exactly happens, when yield and return are used in the same function in Python, like this?

def find_all(a_str, sub):
    start = 0
    while True:
        start = a_str.find(sub, start)
        if start == -1: return
        yield start
        start += len(sub) # use start += 1 to find overlapping matches

Is it still a generator?

nekomimi
  • 1,310
  • 3
  • 10
  • 14

4 Answers4

98

Yes, it' still a generator. The return is (almost) equivalent to raising StopIteration.

PEP 255 spells it out:

Specification: Return

A generator function can also contain return statements of the form:

"return"

Note that an expression_list is not allowed on return statements in the body of a generator (although, of course, they may appear in the bodies of non-generator functions nested within the generator).

When a return statement is encountered, control proceeds as in any function return, executing the appropriate finally clauses (if any exist). Then a StopIteration exception is raised, signalling that the iterator is exhausted. A StopIteration exception is also raised if control flows off the end of the generator without an explict return.

Note that return means "I'm done, and have nothing interesting to return", for both generator functions and non-generator functions.

Note that return isn't always equivalent to raising StopIteration: the difference lies in how enclosing try/except constructs are treated. For example,

>>> def f1():
...     try:
...         return
...     except:
...        yield 1
>>> print list(f1())
[]

because, as in any function, return simply exits, but

>>> def f2():
...     try:
...         raise StopIteration
...     except:
...         yield 42
>>> print list(f2())
[42]

because StopIteration is captured by a bare "except", as is any exception.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
NPE
  • 486,780
  • 108
  • 951
  • 1,012
  • 11
    Do you know what would happen if the `return` had an argument? – zwol Oct 27 '14 at 20:06
  • 21
    @Zack In Python 2.x, it'd be a SyntaxError: `SyntaxError: 'return' with argument inside generator`. It's allowed in Python 3.x, but is primarily meant to be used with coroutines - you make asynchronous calls to other coroutines using `yield coroutine()` (or `yield from coroutine()`, depending on the asynchronous framework you're using), and return whatever you want to return from the coroutine using `return value`. In Python 2.x, you need to use a trick like `raise Return(value)` to return values from coroutines. – dano Oct 27 '14 at 20:08
43

Yes, it is still a generator. An empty return or return None can be used to end a generator function. It is equivalent to raising a StopIteration(see @NPE's answer for details).

Note that a return with non-None arguments is a SyntaxError in Python versions prior to 3.3.

As pointed out by @BrenBarn in comments starting from Python 3.3 the return value is now passed to StopIteration.

From PEP 380:

In a generator, the statement

return value

is semantically equivalent to

raise StopIteration(value)
Community
  • 1
  • 1
Ashwini Chaudhary
  • 244,495
  • 58
  • 464
  • 504
  • 2
    Do you know what would happen if the `return` had an argument (other than `None`)? – zwol Oct 27 '14 at 20:07
  • 9
    In Python 3.3 and up, you can use `return` with an argument to pass the argument to the StopIteration that is raised. See [this question](http://stackoverflow.com/questions/16780002/return-in-generator-together-with-yield-in-python-3-3). – BrenBarn Oct 27 '14 at 20:09
  • 1
    @AshwiniChaudhary The coroutine implementation in the new [`asyncio`](https://docs.python.org/3/library/asyncio-task.html) module is built on that feature (along with the `yield from` keyword). – dano Oct 27 '14 at 20:14
  • @dano All of it started with [PEP 342](http://legacy.python.org/dev/peps/pep-0342/) right? – Ashwini Chaudhary Oct 27 '14 at 20:31
  • 4
    @AshwiniChaudhary That enabled basic coroutines in Python - the ability to send values/exceptions into generators, and receive them via `value = yield`, etc. The introduction of `yield from` and the ability to `return` values from generators came with [PEP 380](http://legacy.python.org/dev/peps/pep-0380/), both of which are leveraged leveraged by `asyncio`. You can still have a robust coroutine implementation with just the features provided by PEP 343, it's just a little less clean to write them. – dano Oct 27 '14 at 20:39
  • It's probably worth explicitly noting this will just silently ignore the returned value in a for-loop, for example, which can be quite counter-intuitive. – Bernhard Barker Dec 26 '19 at 01:56
  • @AshwiniChaudhary saying that `raise StopIteration()` is equivalent to `return` is misleading. If you raise the exception instead of using return, the exception will NOT be catched by the generator function and will be thrown as any other exception. – icaine Sep 30 '21 at 08:16
17

There is a way to accomplish having a yield and return method in a function that allows you to return a value or generator.

It probably is not as clean as you would want but it does do what you expect.

Here's an example:

def six(how_many=None):
    if how_many is None or how_many < 1:
        return None  # returns value

    if how_many == 1:
        return 6  # returns value

    def iter_func():
        for count in range(how_many):
            yield 6
    return iter_func()  # returns generator
William Rusnack
  • 908
  • 8
  • 15
  • not clear "that it does do what you expect". could you provide an example of how your approach could be put to good use? – ShpielMeister Mar 13 '20 at 00:30
  • "that it does do what you expect" as in the subject of the question "Return and yield in the same function". Personally, I have switched to Haskell and this could be used/managed well with an Algebraic Data Type but with Python it is hard enough to manage your types and this does not fit well in the Python type declarations. So, you if you are asking the question of how this could be put to good use please do not use it. Otherwise, this allows you to return no, a single, or multiple values. This could be used to effectively traverse a tree. – William Rusnack Mar 15 '20 at 19:35
1

Note: you don't get StopIteration exception with the example below.

def odd(max):
    n = 0
    while n < max:
        yield n
        n = n + 1
    return 'done'


for x in odd(3):
    print(x)

The for loop catches it. That's its signal to stop

But you can catch it in this way:

g = odd(3)

while True:
    try:
        x = next(g)
        print(x)
    except StopIteration as e:
        print("g return value:", e.value)
        break
Rick
  • 7,007
  • 2
  • 49
  • 79