Python how to partially consume an iterable generator (without `next`)?

Question

I have a class acting as an iterable generator (as per Best way to receive the 'return' value from a python generator) and I want to consume it partially with for loops.

I can't use next (like Python -- consuming one generator inside various consumers) because the first partial consumption uses a library that only takes iterators. How can I continue using the generator starting from where the library function left off?

(Related: Pause Python Generator, Is there a way to 'pause' or partially consume a generator in Python, then resume consumption later where left off?)

class gen(): # https://stackoverflow.com/q/34073370
    def __iter__(self):
        for i in range(20):
            yield i

# I want to partially consume in the first loop and finish in the second
my_gen_2 = gen()
for i in my_gen_2:  # imagine this is the internal implementation of the library function
    print(i)
    if i > 10:      # the real break condition is when iterfzf recieves user input
        break

for i in my_gen_2:  # i'd like to process the remaining elements instead of starting over
    print('p2', i)

# the confusion boils down to this
my_gen = gen()
for i in my_gen:
    print(i)    # prints 1 through 20 as expected
for i in my_gen:
    print('part two', i)    # prints something, even though the generator should have been "consumed"?

napuzba · Accepted Answer · 2020-11-14T19:46:41.357

3

Every time you iterable generator in a loop, you get a new iterator. For example:

class gen(): # https://stackoverflow.com/q/34073370
    def __init__(self):
        self.count = 0
    def __iter__(self):
        self.count += 1
        print("Hallo iter {0}".format(self.count))
        yield self.count

my_gen = gen()
>>> for i in my_gen:
...    pass
Hallo iter 1
>>> for i in my_gen:
...    pass
Hallo iter 2

If you want to use the old iterator, just work with gen().__iter__()

>>> my_gen = gen().__iter__()
>>> for i in my_gen:
...    pass
Hallo iter 1
>>> for i in my_gen:
...    pass

edited Nov 14 '20 at 19:46

answered Nov 14 '20 at 19:36

napuzba

6,033
3
21
32

Oh haha I should have seen that. How would you recommend implementing this then? – Exr0n Nov 14 '20 at 19:38

score 0 · Answer 2 · answered Nov 14 '20 at 19:44

As @napuzba pointed out, __iter__ returns a whole new generator every time it is used. Instead, store the state in self:

class gen():
    def __init__(self):
        self.count = 0
    def __iter__(self):
        while self.count < 20:
            self.count += 1
            yield self.count

Python how to partially consume an iterable generator (without `next`)?

2 Answers2