0

I'm storing data from an API (that I then store as a pickle) and sometimes there are errors due to missing fields. I'm wondering how I can make it so that it only targets the problematic variable and logs its value as "NULL".

The issue is that I'm storing 6 different variables, so if a single one of them has an issue, all other 5 variables will be skipped. Instead, I want that (or those) problematic variables get replaced by the value "NULL" (or None).

meta = loaded_from_pickle('myData.pickle')

def getAllData():

    data = []
    for i in range(len(meta)):
        try:
            name = meta[i]['base']['name']
            weight = meta[i]['base']['weight']
            height = meta[i]['base']['height']

            age = meta[i]['secondary']['age']
            eye_color = meta[i]['secondary']['eye_color']
            birthday = meta[i]['secondary']['birthday']

            data.append((name, weight, height, age, eye_color, birthday))

        except:
            pass

    return data

print(getAllData())

So what happens is that I end up losing a lot of data points because some data doesn't have "eye_color" or whatnot. So what I should do when there's an "except" should be "make problematic variable = "NULL" ", rather than just passing the entire loop.

Faindirnomainzein
  • 351
  • 3
  • 9

1 Answers1

1

Instead of accessing the keys directly using square brackets try using get() it returns a default value of None if the key is not found.

See this answer for more info https://stackoverflow.com/a/11041421/3125312

You could use it in your code:

name = meta[i]['base'].get('name')
weight = meta[i]['base'].get('weight')
height = meta[i]['base'].get('height')
...

You could also rewrite your for loop using enumerate assuming meta is a list

for index, value in enumerate(meta):
    name = value['base'].get('name')
    ...
Siddharth
  • 126
  • 4
  • That works well but what if there's something like meta[i]['base']['fullname']['firstname']? What if fullname can return None and firstname as well? ['base'].get('fullname').get('firstname') doesn't work because I can't "get" from a None value – Faindirnomainzein May 03 '22 at 04:01
  • 1
    `get()` takes in a second argument to use as a default value instead of `None` - use a dict as a default for fullname - `['base'].get('fullname', {}).get('firstname')` – Siddharth May 03 '22 at 06:43