How do I iterate through 2 nested for loops in Python?

Question

I have a script that pulls server info from GCP that works fine:

import googleapiclient.discovery
compute = googleapiclient.discovery.build('compute', 'v1')
project_id = 'company-dev'
zone = 'us-east1-d'
def list_instances(compute,project_id,date):
    zone = 'us-east1-d'
    delete_from_collection(date)
    result = compute.instances().list(project=project_id, zone=zone).execute()
    items = result['items']
    for item in items:
        instance_id = item['id']
        name = item['name']
        timestamp = item['creationTimestamp']
        private_ip = item['networkInterfaces'][0]['networkIP']
        #network = item['networkInterfaces'][0]['network']
        instance_dict = {'Instance ID': instance_id, 'Name': name,'Network Interfaces': private_ip,'Timestamp': timestamp,}
        #print(instance_dict)
        insert_coll(instance_dict,date)

But I need to alter this function so that it also loops through a list of zones. And when I add the zones list and another for loop for the zones list it loses sight of a variable called items = result['items'] that works in the previous version.

This function with the added for loop:

def list_instances(compute,project_id,date):
    delete_from_collection(date)
    zones = ['us-east1-b','us-east1-c','us-east1-d','us-east4-c','us-east4-b','us-east4-a']
    for zone in zones:
        result = compute.instances().list(project=project_id, zone=zone).execute()
        items = result['items']
        for item in items:
            instance_id = item['id']
            name = item['name']
            timestamp = item['creationTimestamp']
            private_ip = item['networkInterfaces'][0]['networkIP']
            #network = item['networkInterfaces'][0]['network']
            instance_dict = {'Instance ID': instance_id, 'Name': name,'Network Interfaces': private_ip,'Timestamp': timestamp,}
            #print(instance_dict)
            insert_coll(instance_dict,date)

Produces the following error:

Traceback (most recent call last):
  File "gcp_list_instances.py", line 379, in <module>
    main()
  File "gcp_list_instances.py", line 370, in main
    list_instances(compute,project_id,date)
  File "gcp_list_instances.py", line 151, in list_instances
    items = result['items']
KeyError: 'items'

Why does the inner for loop lose track of the items = result['items'] variable? And how can I get the inner for loop to recognize that variable?

`result` has no key `items`. That have nothing to do with the inner loop. Print `result` to debug it and see what you got. — Sven Eberth, May 18 '21 at 21:49
Result has key items in the first loop at the top of the post. — bluethundr, May 18 '21 at 21:52
If I print out `result` I see a bunch of output. There are keys and values there. — bluethundr, May 18 '21 at 21:53
`result` has the same output from the 1st version and the 2nd version of the function. That's what's confusing me. — bluethundr, May 18 '21 at 21:54
I would suggest a reproducible example, but it's telling you KeyError: 'items' so ... kind of self explanatory. Just because something else has items doesn't really matter. Maybe it's just one result that is lacking. You can default it to an empty list with result.get('items', []) or if you don't have a debugger you can catch KeyError and print that result. — Kenny Ostrom, May 18 '21 at 21:56
The first block of code has a single `result` value. The second block has a separate value of `result` for each zone in your list. Perhaps they don't all have the same keys? — jasonharper, May 18 '21 at 22:35
Hey!!! That must be it. Some zones have no servers in them. THAT will cause the exception. So perhaps if I wrap JUST that one statement in a try block that will solve the problem? — bluethundr, May 18 '21 at 23:19

score 1 · Accepted Answer · answered May 19 '21 at 14:15

As I mentioned in comments, earlier, you can just provide a default value, so you don't get an exception. And if the empty list is your default, you can still iterate over it. The trick here is "nothing happens" so this code gracefully skips over those errors:

for item in result.get('items', []):
    insert_coll_stuff(item)

docs: https://docs.python.org/3/library/stdtypes.html?highlight=get#dict.get
example: https://stackoverflow.com/a/11041421/1766544

score 0 · Answer 2 · answered May 18 '21 at 23:50

The problem was that there were no servers in some of the zones. So the items variable wouldn't exist in those zones.

Wrapping the code that creates the result variable in a try block solved the problem. This is the final version of the function:

def list_instances(compute,project_id,date):
    delete_from_collection(date)
    items = []
    zones = ['us-east1-b','us-east1-c','us-east1-d','us-east4-c','us-east4-b','us-east4-a','us-central1-c','us-central1-a','us-central1-f','us-central1-b','us-west1-b','us-west1-c','us-west1-a','europe-west4-a','europe-west4-b','europe-west4-c','europe-west1-b','europe-west1-d','europe-west1-c','europe-central2-a','europe-central2-b','europe-central2-c','europe-north1-a','europe-north1-b','europe-north1-c','europe-west6-a','europe-west6-b','europe-west6-c','northamerica-northeast1-a','northamerica-northeast1-b','northamerica-northeast1-c','us-west2-a','us-west2-b','us-west2-c','us-west3-a','us-west3-b','us-west3-c','us-west4-a','us-west4-b','us-west4-c']
    for zone in zones:
        try:
            result = compute.instances().list(project=project_id, zone=zone).execute()
            items = result['items']
        except Exception as e:
            pass
        if items:
            for item in items:
                instance_id = item['id']
                name = item['name']
                timestamp = item['creationTimestamp']
                private_ip = item['networkInterfaces'][0]['networkIP']
                instance_dict = {'Instance ID': instance_id, 'Name': name,'Network Interfaces': private_ip,'Timestamp': timestamp,}
                insert_coll(instance_dict,date)

Would recommend only catching the `KeyError` exception to be specific so you don't mask another problem. You can also do a `continue` in your exception logic — sedavidw, May 19 '21 at 00:01
Ok that's good advice. I will do that. I appreciate the input. — bluethundr, May 19 '21 at 00:20

How do I iterate through 2 nested for loops in Python?

2 Answers2