Conditional checks for JSON key-value pairs being ignored

Question

I am building a point feature class from a web call that returns JSON. The JSON is a bit sketchy in that sometimes keys do not exist in the record. I am trying to do this, once I have a valid JSON object:

#requests stuff above this

for i in jsonObj:
    try:
        if i['properties']['country']:
            country = i['properties']['country']
        else:
            country = 'UNK'
            print('Country name not found, using {}'.format(country))
    except KeyError, e:
        print('Key error: reason: {}'.format(str(e)))
        pass

#then do all the arcpy FC creation stuff

The result is a whole bunch of key errors with "reason: 'country'" and instead of building those rows with the generic 'country' value of 'UNK', it will simply ignore them and build the feature class, leaving out those points.

I have taken out the try and left it as a conditional check, but it fails at the first row that lacks a 'country' key.

In summary, I'm just trying to check if a key-value pair exists; if it doesn't, assign a generic value of 'UNK' to the country variable.

It seems like part of the problem might be that if i['properties']['countries'] is checking for a value, but not the existence of the key itself? How might I more efficiently check for the existence of the key?

I have read Check if a given key already exists in a dictionary and have modified my code to both of these, and neither yield the expected outcome:

for i in jsonObj:
    try:
        # get coordinates first
        if i['geometry']['coordinates']:
            ycoord = float(i['geometry']['coordinates'][1])
            xcoord = float(i['geometry']['coordinates'][0])
            if i['properties']['city'] in i:
                city = i['properties']['city']
            else:
                city = 'UNK'

            if i['properties']['country'] in i:
                country = i['properties']['country']
            else:
                country = 'UNK'

and

for i in jsonObj:
    try:
        # get coordinates first
        if i['geometry']['coordinates']:
            ycoord = float(i['geometry']['coordinates'][1])
            xcoord = float(i['geometry']['coordinates'][0])
            if 'city' in i:
                city = i['properties']['city']
            else:
                city = 'UNK'

            if 'country' in i:
                country = i['properties']['country']
            else:
                country = 'UNK'

I do have the 'properties' key in every record/dictionary, but whether I have a 'country' key is not guaranteed. Some rows in the json response have it, some rows don't

you may try `if 'properties' in i:, not `if 'country' in i:`. — Jean-François Fabre, Mar 03 '17 at 21:10
Yes, I do have the 'properties' key in every record/dictionary, but whether I have a 'country' key is not guaranteed. Some rows in the json response have it, some rows don't. — auslander, Mar 03 '17 at 21:16

Jean-François Fabre · Accepted Answer · 2017-03-03T21:18:23.230

2

Your last try:

if 'country' in i:
    country = i['properties']['country']
else:
    country = 'UNK'

was close, but you're managing a dict of dicts, and 'country' has better chance to be a key of the sub-dict, so the fix would be:

if 'country' in i['properties']:
    country = i['properties']['country']
else:
    country = 'UNK'

or even better & shorter using get with a default value (I recommend that last one over the quickfix):

country = i['properties'].get('country','UNK')

edited Mar 03 '17 at 21:18

answered Mar 03 '17 at 21:13

Jean-François Fabre

137,073
23
153
219

Moving to the .get() method was going to be my next mod to this script, so thank you for the clarity on it. So with that, I could eliminate even doing an if-else check at all, yes? – auslander Mar 03 '17 at 21:21

score 0 · Answer 2 · answered Mar 03 '17 at 21:57

It seems like you don't fully understand json implementation.

x = i['geometry']['coordinates'] is basically y = i['geometry']; x = y['coordinates'], so you need safety check for each layer, becaue i['geometry'] not only will throw exception when 'geometry' field is not found but also the returned object must also implement [] for ['coordinates'] to work (so in this case it must be another json object, and not string,bool,None etc.)

I also believe your json object is implemented using python dictionary, so you can check if certain field, e.g. 'geometry' exists using x.get('geometry'), which will return either its value or None object. You can also use city = x.get('city', 'UKN') to set default value (json object, string, bool, None etc.) if it was not found in dict (python dict.get method implements default handler).

So in the end you should have something like this:

geo = i.get('geometry')
if not geo: return

coords = geo.get('coordinates')
if not coords: return
xcoord, ycoord = float(coords[0]), float(coords[1])

props = i.get('properties')
if not props: return
city = props.get('city', 'UNK')
country = props.get('country', 'UNK')

This is a draft and I did not test it, also this strongly asserts that your json object is based on python dict object.

You're correct, I'm an infant with json. Thanks for the tips. I'm not at my usual workstation, but I'll be glad to try this out in a couple of days and follow up. — auslander, Mar 04 '17 at 23:52
I ran through this this morning and it worked the same as above, but with better error handling. Thanks for the tip! — auslander, Mar 06 '17 at 15:44

Conditional checks for JSON key-value pairs being ignored

2 Answers2