Append on a 3rd Order List Not Working as Expected

Question

I have some code that takes n groups of states and then creates multiple "neighbors" by swapping states between groups randomly.

from random import randint as rand

groups = [['VT', 'TX', 'HI', 'AR', 'PA', 'OK', 'AK', 'MI', 'ND', 'ID', 'NY', 'WV', 'FL', 'UT'],
          ['MO', 'MA', 'SD', 'NE', 'GA', 'MN', 'SC', 'CA', 'RI', 'AZ', 'NH', 'MS', 'AL', 'WI', 'GU', 'CT', 'MT', 'TN', 'OH', 'OR', 'IA'],
          ['NV', 'KY', 'NM', 'KS', 'IN', 'LA', 'DE', 'MD', 'CO', 'IL', 'VA', 'NC', 'WY', 'NJ', 'WA', 'ME']]

def hill_climb_nbr(groups,nbr_size):
    #swap one state in and out of each group

    group_list = []

    for g in range(0,nbr_size):

        temp_val = []

        groups_copy = groups

        for i in range(0,len(groups)):
            rand_num = rand(0,len(groups[i])-1)
            temp_val.append(groups[i][rand_num])
            groups_copy[i].pop(rand_num)
            print(rand_num)

        for j in range(0,len(temp_val)):
            groups_copy[j].append(temp_val[j-1])

        print(groups_copy)
        print('group_list')

        group_list.append(groups_copy)

        print(group_list)

    return group_list

hill_climb_nbr(groups,3)

Here is an example of input:

[['VT', 'TX', 'HI', 'AR', 'PA', 'OK', 'AK', 'MI', 'ND', 'ID', 'NY', 'WV', 'FL', 'UT'],
 ['MO', 'MA', 'SD', 'NE', 'GA', 'MN', 'SC', 'CA', 'RI', 'AZ', 'NH', 'MS', 'AL', 'WI', 'GU', 'CT', 'MT', 'TN', 'OH', 'OR', 'IA'],
 ['NV', 'KY', 'NM', 'KS', 'IN', 'LA', 'DE', 'MD', 'CO', 'IL', 'VA', 'NC', 'WY', 'NJ', 'WA', 'ME']]

The output of hill_climb_nbr(groups,3) where groups is the list of lists shown above is this:

[[['VT', 'TX', 'HI', 'AR', 'PA', 'OK', 'AK', 'MI', 'ND', 'ID', 'NY', 'WV', 'FL', 'UT'],
  ['MO', 'MA', 'SD', 'NE', 'GA', 'MN', 'SC', 'CA', 'RI', 'AZ', 'NH', 'MS', 'AL', 'WI', 'GU', 'CT', 'MT', 'TN', 'OH', 'OR', 'IA'],
  ['NV', 'KY', 'NM', 'KS', 'IN', 'LA', 'DE', 'MD', 'CO', 'IL', 'VA', 'NC', 'WY', 'NJ', 'WA', 'ME']],
 [['VT', 'TX', 'HI', 'AR', 'PA', 'OK', 'AK', 'MI', 'ND', 'ID', 'NY', 'WV', 'FL', 'UT'],
  ['MO', 'MA', 'SD', 'NE', 'GA', 'MN', 'SC', 'CA', 'RI', 'AZ', 'NH', 'MS', 'AL', 'WI', 'GU', 'CT', 'MT', 'TN', 'OH', 'OR', 'IA'],
  ['NV', 'KY', 'NM', 'KS', 'IN', 'LA', 'DE', 'MD', 'CO', 'IL', 'VA', 'NC', 'WY', 'NJ', 'WA', 'ME']],
 [['VT', 'TX', 'HI', 'AR', 'PA', 'OK', 'AK', 'MI', 'ND', 'ID', 'NY', 'WV', 'FL', 'UT'],
  ['MO', 'MA', 'SD', 'NE', 'GA', 'MN', 'SC', 'CA', 'RI', 'AZ', 'NH', 'MS', 'AL', 'WI', 'GU', 'CT', 'MT', 'TN', 'OH', 'OR', 'IA'],
  ['NV', 'KY', 'NM', 'KS', 'IN', 'LA', 'DE', 'MD', 'CO', 'IL', 'VA', 'NC', 'WY', 'NJ', 'WA', 'ME']]]

It is a strain to see, but it is just the same list of lists repeated three times.

While trying to diagnose the problem, I found that groups_copy looks correct right before the group_list.append(groups_copy). Somehow the final group_list turns out to be the last groups_copy repeated nbr_size times instead of each iteration of groups_copy.

I've been looking at this for a few hours now and I'm about to loose my mind. Does anything stand out to you as to where the issue is?

@Prune I believe this is an MCVE since I supplied the function code and the inputs that I'm using. I made some slight edits to make it easier to copy and paste. I wish I could be more specific on the nature of the problem, but I don't know why it is giving the results it is. — Jarom, Feb 13 '19 at 22:39
@jarom I'd say the `groups_copy` is your problem. Try `groups_copy = copy.copy(groups)`. I'm not in a position to run your code right now, but that's what jumps out at me. — John Keyes, Feb 13 '19 at 22:45
Just take note, you will need to `from copy import copy` for the above — Jab, Feb 13 '19 at 22:46
And @JohnKeyes copy and deepcopy both return different results — Jab, Feb 13 '19 at 22:49

score 2 · Accepted Answer · answered Feb 13 '19 at 22:49

It's still not a minimal example, but it's close enough to debug.

Your problem is that your line

groups_copy = groups

does not copy the list -- it merely copies the reference to the top level. Even the usual solution of

groups_copy = groups[:]

won't fix it, because this is a nested list -- the slice merely copies the top level.

You need a deep copy:

from random import randint as rand
from copy import deepcopy

groups = [['VT', 'TX', 'HI'],
          ['MO', 'MA', 'SD'],
          ['NV', 'KY', 'NM'],
         ]

...

    groups_copy = deepcopy(groups)

Output:

0
0
1
[['TX', 'HI', 'KY'], ['MA', 'SD', 'VT'], ['NV', 'NM', 'MO']]
group_list
[[['TX', 'HI', 'KY'], ['MA', 'SD', 'VT'], ['NV', 'NM', 'MO']]]
2
0
1
[['VT', 'TX', 'KY'], ['MA', 'SD', 'HI'], ['NV', 'NM', 'MO']]
group_list
[[['TX', 'HI', 'KY'], ['MA', 'SD', 'VT'], ['NV', 'NM', 'MO']], [['VT', 'TX', 'KY'], ['MA', 'SD', 'HI'], ['NV', 'NM', 'MO']]]
2
2
1
[['VT', 'TX', 'KY'], ['MO', 'MA', 'HI'], ['NV', 'NM', 'SD']]
group_list
[[['TX', 'HI', 'KY'], ['MA', 'SD', 'VT'], ['NV', 'NM', 'MO']],
 [['VT', 'TX', 'KY'], ['MA', 'SD', 'HI'], ['NV', 'NM', 'MO']],
 [['VT', 'TX', 'KY'], ['MO', 'MA', 'HI'], ['NV', 'NM', 'SD']]]

I inserted line feeds into the final line to make the result easier to read.

Definitely `deepcopy`, tested and seems to give expected results — Jab, Feb 13 '19 at 22:52

Append on a 3rd Order List Not Working as Expected

1 Answers1