Python, how can I set iterator of enumerate to 0 while reading same file again and again

Question

with open("...txt") as fp: 
    for i, line in enumerate(fp): 
        if some condition : 
            i=0
            fp.seek(0)

Text is huge, GBs of data so I use enumerate. I need to process this huge file several thousands of time so I decided to open it just at first time for efficiency. However although this code works, i does not become 0 and it just goes on incrementing. I need that to be zero because I need position of lines i. And it is just inefficient to multiply billions*several thousands everytime and make some modular arithmetic.

So my question is how can I set i to be zero when I go back to the beginning of file? Thanks in advance (I use python 3.6)

You may want to look into doing a nested loop rather than a loop with them on the same line if you want them to count separately — JoshKopen, Jun 19 '17 at 14:17
If you want to reset `i` to zero part way through your loop, don't use `enumerate`. Increment and reset `i` manually. — khelwood, Jun 19 '17 at 14:17
If you are looking to read very large files, you can just iterate over each line per this: https://stackoverflow.com/questions/6475328/read-large-text-files-in-python-line-by-line-without-loading-it-in-to-memory. Furthermore, — idjaw, Jun 19 '17 at 14:18
however i tested some speed and reading every line one by one is slower and this enumerate is faster. Isn't there any way to set it to 0? — xcvbnm, Jun 19 '17 at 14:21
You should also test the current approach and closing the file instead of seeking. — Andras Deak -- Слава Україні, Jun 19 '17 at 14:23
code is already ready with closing and opening everytime and it is just slow. So i want to test with going back to beginning approach and this question is actually for testing how it will improve speed. But i am stuck in that 'i' — xcvbnm, Jun 19 '17 at 14:26
enumerate is a generator function. It's "exhausted" by the time it completes. This is also why you can't just "reset" it. Here is the [PEP](https://www.python.org/dev/peps/pep-0279/) on enumerate. — idjaw, Jun 19 '17 at 14:29
ok thank you, if you write as answer i will accept it. and thank you all for answers — xcvbnm, Jun 19 '17 at 14:31

score 3 · Answer 1 · answered Jun 19 '17 at 14:51

You could always make your own resettable enumerator, but there are probably better ways to do what you really want to do.

Still, here's what a resettable enumerator looks like:

 def reset_enumerate(thing, start=0):
     x = start
     for t in thing:
         val = yield t, x
         if val is not None:
             x = val
         else:
             x += 1

Then you would use it like this:

r = reset_enumerate(range(10))
for i, num in r:
    print('i:', i, 'num:', num)     
    if i == 5:
        i, num = r.send(0)
        print('i:', i, 'num:', num)

score 2 · Accepted Answer · answered Jun 19 '17 at 14:39

As stated in the comment, enumerate is a generator function. It's "exhausted" by the time it completes. This is also why you can't just "reset" it. Here is the PEP on enumerate to further explain how it works.

Furthermore, as also indicated in the comments, this post provides the typical way to handle large files.

score 1 · Answer 3 · answered Jun 19 '17 at 14:32

Here is an example of how you can emulate a scenario like yours:

Assuming i have a file called input.txt with this kind of data:

1
2
3

Code:

j = 0
with open('input.txt', 'r') as f:
    for k in f:
        # A break condition
        # If not we'll face an infinite loop
        if j > 4:
            break
        if k.strip() == '2':
            f.seek(0)
            print("Return to position 0")
            # Don't forget to increment j 
            # Otherwise, we'll end up with an infinite loop
            j += 1
        print(k.strip())

Will output:

1
Return to position 0
2
1
Return to position 0
2
1
Return to position 0
2
1
Return to position 0
2
1
Return to position 0
2

Python, how can I set iterator of enumerate to 0 while reading same file again and again

3 Answers3