Python for loop is not looping through all elements

Question

How is it possible that my Python code is not looping through all elements in my list? I have code like this:

        # Get only the XML and XLSX files
        filesInSFTP = [file for file in filesInSFTP if file.lower().endswith(('.xml', '.xlsx'))]
        print("Before Sorted: " + str(filesInSFTP))
        print(len(filesInSFTP))
        for filename in filesInSFTP:
            print("file name in loop: " + str(filename))
            if filename.endswith('.xlsx'):
                if 'EE Nav_Plan Mapping' not in filename:
                    filesInSFTP.remove(filename)
                    print("removed " + filename)

        # order the list so that XLSX comes first, before XML
        filesInSFTP = sorted(filesInSFTP, key=lambda x: os.path.splitext(x)[1])
        print("After Sorted: " + str(filesInSFTP))

And this the output from the print statements:

Before Sorted: ['Ease_Plan_Mappings.xlsx', 'EE Nav_Plan Mapping.xlsx', 'advanced.xlsx', 'Manco.xlsx']
4
file name in loop: Ease_Plan_Mappings.xlsx
removed Ease_Plan_Mappings.xlsx
file name in loop: advanced.xlsx
removed advanced.xlsx
After Sorted: ['EE Nav_Plan Mapping.xlsx', 'Manco.xlsx']

As you can see, the print statements are not printing "file name in loop: Manco.xlxs" or "file name in for loop: EE Nav_Plan Mapping.xlsx" Why is this happening? I need my for loop to go through the entire list which is printed right before the loop.

EDIT: If my code were working correctly then "manco.xlsx" would have been removed from the list because it does not have "EE nav_Plan Mapping" in its name.

TL;DR: `filesInSFTP.remove(filename)` is messing things up. Don't remove from a list that you're currently iterating. — Carcigenicate, May 04 '21 at 14:09
@Carcigenicate is basically correct. a quickfix would be to change your iteration line from `for filename in filesInSFTP:` to `for filename in list(filesInSFTP):` as that would basically create a copy of your list, and the remove wouldn't affect this copy — oskros, May 04 '21 at 14:12

kengru · Answer 1 · 2021-05-04T14:20:29.167

0

As others have commented, filesInSFTP.remove(filename) is actually modifying the list that you are iterating over. Since you remove an element before the iteration is finished its skipping one element in the middle.

One solution might be to copy the list just for iterating and then remove the elements from the list you are going to use.

edited May 04 '21 at 14:20

answered May 04 '21 at 14:14

kengru

89
8

3

It's a list, not an array. They're [different things in Python](https://learnpython.com/blog/python-array-vs-list/). – jarmod May 04 '21 at 14:17
That is actually what I ended up doing – mastercool May 04 '21 at 14:19
@jarmod, you are correct. I edited the post, thanks for the clarification. – kengru May 04 '21 at 17:52

score 0 · Accepted Answer · answered May 04 '21 at 14:20

0

I ended up fixing this by copying the list just before the iteration.

        li = filesInSFTP.copy()
        for filename in li:
            print("file name in loop: " + str(filename))
            if filename.endswith('.xlsx'):
                if 'EE Nav_Plan Mapping' not in filename:
                    filesInSFTP.remove(filename)
                    print("removed " + filename)

answered May 04 '21 at 14:20

mastercool

463
12
35

if your solution is basically just doing what others comment / answer with, there is no need to post the solution as a separate answer – oskros May 04 '21 at 14:21
figured its still useful to see the code for people like myself – mastercool May 04 '21 at 14:27

Python for loop is not looping through all elements

2 Answers2