Python from generator to list

Question

I've seen some examples where we can transfer generator into list as below.

First example:

print [2 * n for n in range(5)]
# same as the list comprehension above
print list(2 * n for n in range(5))

Second example:

def double(L):
    for x in L:
        yield x*2

# eggs will be a generator
eggs = double([1, 2, 3, 4, 5])
# the above is equivalent to ("generator comprehension"?)
eggs = (x*2 for x in [1, 2, 3, 4, 5])
# need to do this if you need a list
eggs = list(double([1, 2, 3, 4, 5]))
print eggs
# the above is equivalent to (list comprehension)
eggs = [x*2 for x in [1, 2, 3, 4, 5]]
print eggs

My question is, can all generators be transferred into list?(I failed at below example):

def get_primes(number):
    while True:
        if is_prime(number):
            number = yield number
        number += 1

def is_prime(number):
    if number > 1:
        if number == 2:
            return True
        if number % 2 == 0:
            return False
        for current in range(3, int(math.sqrt(number) + 1), 2):
            if number % current == 0:
                return False
        return True
    return False

generator = get_primes(5)
print list(generator)

Output:TypeError: unsupported operand type(s) for +=: 'NoneType' and 'int'

The problem is here:number = yield number for some reason. Some explanation on this are appreciated.

What? `number = yield number`? `yield` doesn't return anything. — Remi Guan, Jan 10 '16 at 09:05
Don't edit it, because that line is why you're getting that TypeError. — user2357112, Jan 10 '16 at 09:07
`yield` actually can [return a value in a way](http://stackoverflow.com/questions/10695456/what-is-the-result-of-a-yield-expression-in-python). — TigerhawkT3, Jan 10 '16 at 09:08
@ZhaoWu Looks like you are trying to form an infinite generator. Making a list out of this will cause `MemoryError` (system running out of memory) — shanmuga, Jan 10 '16 at 09:09
@user2357112: true...Thanks...And do you mind a brief explanation? — Zhao Wu, Jan 10 '16 at 09:09

user2357112 · Accepted Answer · 2016-01-10T09:17:37.283

Your generator is broken:

number = yield number

This should just be yield number. Assigning the value of a yield expression to a variable is only useful when you expect the caller to send values into the generator. When you just iterate over a generator normally, all its yield expressions evaluate to None. This line assigns None to number, and then number += 1 TypeErrors because you're trying to add an integer to None.

If you had tried to iterate over this generator with a for loop, you would have gotten the same error.

That said, not all generators can be converted to a list, and your code, either fixed or unfixed, is an example of why: a generator might throw an exception or yield values forever. The list constructor is roughly equivalent to

def list(arg):
    l = []
    for item in arg:
        l.append(arg)
    return l

If the generator throws an exception, the exception propagates out of the list constructor and terminates the loop. If the generator yields forever, the loop goes forever, or at least until you run out of memory or patience. You could also have a generator that refuses to yield:

def noyield():
    while True:
        pass
    yield 1  # Not happening.

This is such a quick and perfect answer. – Zhao Wu Jan 10 '16 at 09:17 — Zhao Wu, Jan 10 '16 at 09:17

hiro protagonist · Answer 2 · 2016-01-10T09:18:40.220

there are 2 issues here:

number = yield number

will set number to None (as you do not send anything to the generator).

the second issue is: your generator never terminates. if you generate a list from that python probably run into a memory overflow.

this is what you could do:

import math

def get_primes(start, stop):
    n = start
    while True:
        if n >= stop:
            raise StopIteration
        if is_prime(n):
            yield n
        n += 1

def is_prime(number):
    # no changes here

generator = get_primes(5, 15)
print list(generator)  # [5, 7, 11, 13]

any generator that does not raise an exception other than StopIteration and terminates can be converted to a list.

score 1 · Answer 3 · answered Jan 10 '16 at 09:16

Change

number += 1

to

number = number + 1 if (number is not None) else 1

In that way you would only update number if something sent to it and avoid exception mentioned above. But you would override number if you won't sent value to generator.

But two issues described by @hiro protagonist still exist!

Python from generator to list

3 Answers3