KeyError: 'AvailabilityZone'

Question

Please help me get around this issue: When I try to run the following script, it always returns this error: KeyError: 'AvailabilityZone'

client.describe_instances(Filters=[ {'Name': 'instance-state-name', 'Values': [ 'running', ] } , ], )

Also, I observed that there is no AvailabilityZone string in the response.

Example snippet:

import boto3
import collections
ec2 = boto3.resource('ec2')
instances = ec2.instances.filter(
    Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])


running_instances = {}
for instance in instances:
    az = instance.placement['AvailabilityZone']
    os = instance.platform
    instance_type = instance.instance_type
    if os is None:
        os = 'Linux/UNIX'
    else:
        os = 'Windows'
    if az not in running_instances:
        running_instances[az] = {}
    if os not in running_instances[az]:
        running_instances[az][os] = {}
    if instance_type not in running_instances[az][os]:
        running_instances[az][os][instance_type] = 0
    running_instances[az][os][instance_type] += 1

client = boto3.client('ec2')
response = client.describe_reserved_instances(
    Filters=[
        {
            'Name': 'state',
            'Values': [
                'active',
            ]
        }
    ],
)
cnt = collections.Counter()

reserved_instances = {}
for instance in response['ReservedInstances']:
    az = instance['AvailabilityZone']
    os = instance['ProductDescription']
    instance_type = instance['InstanceType']
    instance_count = instance['InstanceCount']
    if az not in reserved_instances:
        reserved_instances[az] = {}
    if os not in reserved_instances[az]:
        reserved_instances[az][os] = {}
    if instance_type not in reserved_instances[az][os]:
        reserved_instances[az][os][instance_type] = 0
    reserved_instances[az][os][instance_type] += instance_count

print("****** Running Instances breakdown *****")
print(running_instances)
print("****** Reserved Instance breakdown *****")
print(reserved_instances)

Answer 1

First of all, I would recommend pasting your code sample directly into the question rather than linking to it as above. If the linked data ever goes away it will make the question much harder to understand.

I think the problem you are seeing is that as your are looping through the instances returned by describe_instances you are assuming that the placement attribute will always contain a key called AvailabilityZone however if the instance in question is in a stopped state or is terminated that will not be true.

So, you should check to make sure placement has that attribute before accessing it (e.g. if 'AvailabilityZone' in instance.placement).

Answer 2

This is the complete ec2.describe_instances response results. The whole json dict structure ls shown clearly as

# ec2.client.describe_instances() responnse structure
{ 'Reservations':[
   {"Group": [{....}]},  
   {"Instances:[{.....}]} 
   ],
  'NextToken': 'string'
}

ec2.client.describe_reserved_instances is about reserved instances you purchase. Reserved instances discounts are dynamically matched with instances that you start, in hourly basis. So you will never get info like OS, instance count,etc.

# describe_reserved_instances is about reserved instances you purchase
{
    'ReservedInstances': [
        {
            'AvailabilityZone': 'string',
            'Duration': 123,
            'End': datetime(2015, 1, 1),
            'FixedPrice': ...,
            'InstanceCount': 123,
            'InstanceType':  
             ............
        },
    ]
} 

What you need to do is follow the correct python dictionary reference. Python will NOT convert the dictionary to dot notation unless you load module like jsontree(only in python2) to convert them. There is also named tuple conversion but I ddint' use them. It is much easier just use the simple dict reference.

So the correct notation reference should looks like this

result = ec2.instances.filter(
    Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])

for reservation in result['Reservations']:
    for instance in reservation['Instances']:
        az = instance['Placement']['AvailabilityZone']

(update) A new AWS admin/user will raise the question, why describe_instances and describe_reserved_intances show different results set?

The short answer is : they are not the same things. describe_instances will show actual AWS VM that you created in the system. On the other hand, reserved instances is NOT a VM, but actually a discounted billing mechanism. So you will NOT see any VM attribute associated with it, but monetary info. AWS will apply the discount logic whenever it see fits, so you will not see a solid itemised billing from describe_reserved_instances.