dict changed size during iteration, but not editing it in Python

Question

I'm getting the dict changed size during iteration error in Python even though I'm not altering the nested dict. I'm deleting from the original, not the copy that I'm looping through

test = arr.copy()
for map in test:
    for mID in test[map]:  #here is dictionary changed size during iteration
        v = test[map][mID]['val']
        if v < 1:
            del arr[map][mID]

Am I overlooking something?

With nested data structures you need to use `copy.deepcopy()` instead of `copy()` — ettanany, Aug 16 '17 at 12:07
There's no need to copy the whole dict - just iterate over a copy of the keys (i.e. use `list(test)` and `list(test[map])`). — ekhumoro, Aug 16 '17 at 12:18

Jean-François Fabre · Accepted Answer · 2017-08-16T13:02:39.050

you're copying the dictionary but using shallow copy.

Since you're iterating on a sub-dict (which isn't copied, the reference is the same between arr and test), you get this error.

Use the deepcopy function of the copy module like this:

import copy
test = copy.deepcopy(arr)

However, deep-copying an object is overkill for such a problem. You could just iterate on a copy of the items like this (also: always iterate on key+values so you don't have to access the value by key within the loop):

for map,map_values in arr.items():
    for mID,subdict in list(map_values.items()):  # make a copy/force iteration using `list`
        v = subdict['val']
        if v < 1:
            del map_values[mID]

score 1 · Answer 2 · answered Aug 16 '17 at 12:07

1

If you're working with nested dicts, you should create a deep copy.

import copy
test = copy.deepcopy(arr)

Note that copy.copy and dict.copy only create shallow copies!

Docs for dict.copy():

Return a shallow copy of the dictionary.

answered Aug 16 '17 at 12:07

Mike Scotty

10,530
5
38
50

score 1 · Answer 3 · answered Aug 16 '17 at 12:42

The other answers are right: copy() returns a shallow copy of the dictionary, so arr[map] will be a copy of test[map], but arr[map][mID] and arr[map][mID] will be exactly test[map][mID].

The following schema shows the situation:

arr  -+-> 'map1' -+-> { 'id1':{'val':0}, ... }
      |           |
      +-> 'map2' ---+-> { 'id1':{'val':2}, ... }
                  | |
test -+-> 'map1' -+ |
      |             |
      +-> 'map2' ---+

When you try to delete arr['map1']['id1'], you are also trying to delete test['map1']['id1'], yet you are iterating on test['map1']. That's why you are getting this error (the size of test['map1'] may not change while you are iterating on it).

But I don't think that a deepcopy is the good solution. All that you need is to copy the keys of arr[map] and iterate over this copy.

for map in arr:
    for mID in list(arr[map]): # here's the copy of the keys
        v = arr[map][mID]['val']
        if v < 1:
            del arr[map][mID] # allowed because you are not iterating on the dict, but on the copy of the keys

Try it online!

Some reading: How to avoid "RuntimeError: dictionary changed size during iteration" error?

dict changed size during iteration, but not editing it in Python

3 Answers3