Delete unnecessary elements in json

Question

I have the JSON data, where I would like to delete unnecessary elements. I am using element.pop(), and this is normal what we do. But I am wondering if I have more than thousands elements in JSON and my requirements is only for element0, element1 and element6, then again I need to use element.pop().

Do we have something's, where I do not need to pop all unnecessary elements? Because, if my requirement is only for 3 elements and then I need to pop thousands elements. This will take time.

data.json

[
  {
    "fields": {
      "element0": "val0",
      "element1": "val1",
      "element2": "val2",
      "element3": "val3",
      "element4": "val4",
      "element5": "val5",
      "element6": "val6"
    }
  },
  {
    "fields": {
      "element0": "val0",
      "element1": "val1",
      "element2": "val2",
      "element3": "val3",
      "element4": "val4",
      "element5": "val5",
      "element6": "val6"
    }
  }
]

python code

import json
with open('data.json','r') as f:
    s = f.read()
    data = json.loads(s)

### my expectation
for element in data:
    element['fields'].pop('element2', None)
    element['fields'].pop('element3', None)
    element['fields'].pop('element4', None)
    element['fields'].pop('element5', None)

expected.json

[
  {
    "fields": {
      "element0": "val0",
      "element1": "val1",
      "element6": "val6"
    }
  },
  {
    "fields": {
      "element0": "val0",
      "element1": "val1",
      "element6": "val6"
    }
  }
]

why are you doing `s = s.replace('\'','\"')`??? Note, your "json" is **not json at all**. You should probably just fix that — juanpa.arrivillaga, Jul 14 '21 at 08:51
In any case, I suppose, you could just *choose the element you want* instead of popping the elements you don't want... not sure if that's what you are asking. — juanpa.arrivillaga, Jul 14 '21 at 08:55
Because of `JSONDecodeError: Expecting property name enclosed in double quotes`. Yes, that's why I am replacing ' with ". [link][1] [1]: https://stackoverflow.com/questions/39491420/python-jsonexpecting-property-name-enclosed-in-double-quotes — dtc348, Jul 14 '21 at 08:55
Well, precisely, *but that means it was never JSON*. In any case, that is totally unnecessary for the question? Why provide it like that? You should be fixing the file, not adding this extra element of parsing. — juanpa.arrivillaga, Jul 14 '21 at 08:56

juanpa.arrivillaga · Accepted Answer · 2021-07-14T09:18:28.557

You could do something like the following, if you only need certain elements from data:

>>> data = [{'fields': {'element0': 'val0',
...    'element1': 'val1',
...    'element2': 'val2',
...    'element3': 'val3',
...    'element4': 'val4',
...    'element5': 'val5',
...    'element6': 'val6'}},
...  {'fields': {'element0': 'val0',
...    'element1': 'val1',
...    'element2': 'val2',
...    'element3': 'val3',
...    'element4': 'val4',
...    'element5': 'val5',
...    'element6': 'val6'}}]

Then just select the fields you need, then grab from the nested structure:

>>> NEEDED = "element0", "element1", "element6"
>>> for element in data:
...     fields = element['fields']
...     element['fields'] = {n: fields.get(n) for n in NEEDED}
...

Data will then be:

>>> data
[{'fields': {'element0': 'val0', 'element1': 'val1', 'element6': 'val6'}}, {'fields': {'element0': 'val0', 'element1': 'val1', 'element6': 'val6'}}]

Or just re-create data:

>>> NEEDED = "element0", "element1", "element6"
>>> data2 = [{'fields': {n: ele['fields'].get(n) for n in NEEDED}} for ele in data]

This is really nice answer. I have more requirements, how can we bring `element0 `, `element1 ` and `element6 ` along with its values? my expected output like [{"fields":{"element0":"val0","element1":"val1","element6":"val6"}},{"fields":{"element0":"val0","element1":"val1","element6":"val6"}}] — dtc348, Jul 14 '21 at 09:13

Delete unnecessary elements in json

1 Answers1