Remove empty dicts in nested dictionary with recursive function

Question

I'm trying to remove non-values from a nested dictionary. My first effort works fine, but unfortunately keys pointing to now empty dicts still persist.

So if i do:

pass1 = stripper(my_dict)
return stripper(pass1)

This works, but i'm thinking a more elegant nested solution might be possible?

def stripper(self, data):
    if isinstance(data, dict):
        d = ({k: stripper(v) for k, v in data.items()
             if v not in [u'', None]})
        if d:
            return d
    else:
        return data

Edit:

Failing example, dict below returns as {'foo': 'bar', 'bar': None}:

{
    'foo': 'bar',
    'bar': {
        'foo': None,
        'one': None
    }
}

you usually dont pass my_dict to strippers... that doesnt end well — Joran Beasley, Nov 04 '15 at 18:28
Can you post a very simple example where this doesn't work? If it leaves empty dicts on the first pass, then it seems likely that subsequent passes could leave keys with empty values as well... — mgilson, Nov 04 '15 at 18:31

Oliver Dain · Accepted Answer · 2015-11-04T22:22:09.377

11

The dict comprehension is certainly concise but if you expand it out, the solution becomes more obvious:

def stripper(self, data):
    new_data = {}
    for k, v in data.items():
        if isinstance(v, dict):
            v = stripper(v)
        if not v in (u'', None, {}):
            new_data[k] = v
    return new_data

edited Nov 04 '15 at 22:22

answered Nov 04 '15 at 18:31

Oliver Dain

9,617
3
35
48

Was missing the `{}` in the `not v in ('', None, {})` line - I hadn't tested it. Added that and it appears to me to work. Whomever down-voted, take another look as I believe this does the trick. – Oliver Dain Nov 04 '15 at 23:44
Flake8 nags me to say, that third to last line should be `if v not in (u'', None, {}):` – chrisinmtown Feb 27 '23 at 23:25

score 4 · Answer 2 · answered Jun 02 '20 at 19:17

To extend the accepted answer to include objects that might be an element of a list:

def strip_empties_from_list(data):
    new_data = []
    for v in data:
        if isinstance(v, dict):
            v = strip_empties_from_dict(v)
        elif isinstance(v, list):
            v = strip_empties_from_list(v)
        if v not in (None, str(), list(), dict(),):
            new_data.append(v)
    return new_data


def strip_empties_from_dict(data):
    new_data = {}
    for k, v in data.items():
        if isinstance(v, dict):
            v = strip_empties_from_dict(v)
        elif isinstance(v, list):
            v = strip_empties_from_list(v)
        if v not in (None, str(), list(), dict(),):
            new_data[k] = v
    return new_data

To use:

data = {
  'None': None,
  'empty_list': [],
  'empty_dict': {},
  'empty_string': '',
  'list_with_empties': ['', {}, [], None, {'more_empties': '', 'and_even_more': [''], 'one_thing_i_care_about': 'hi'}]
}
stripped_data = strip_empties_from_dict(data)
print(stripped_data)

Sumit Badsara · Answer 3 · 2023-03-02T12:05:19.430

1

Answer of @Oliver is correct, but having some edge cases checks won't hurt, here you go: (Was unable to edit Oliver's answer as the queue was full)

    def dictionary_stripper(data):
        new_data = {}
        
        # Only iterate if the given dict is not None
        if data:
            for k, v in data.items():
                if isinstance(v, dict):
                    v = dictionary_stripper(v)
                
                # ideally it should be not in, second you can also add a empty list if required
                if v not in ("", None, {}, []):
                    new_data[k] = v
            
            # Only if you want the root dict to be None if empty
            if new_data == {}:
                return None
            return new_data
        return None

edited Mar 02 '23 at 12:05

answered Jul 09 '21 at 22:50

Sumit Badsara

769
9
23

CampaignAdminUtils? – chrisinmtown Feb 28 '23 at 01:07
@chrisinmtown Thanks for pointing this out. Left the class name by mistake, ideally that should be recursion, so it should be just `dictionary_stripper` – Sumit Badsara Mar 02 '23 at 12:07

score 0 · Answer 4 · answered Feb 28 '23 at 12:01

I landed here with a need to drop None values yet preserve empty dicts and lists. Reused/modified @OliverDain answer, thanks for that. Hope this helps someone.

def drop_none_values(data):
    """
    Recursively remove None values in a dictionary or list;
    this includes list of dicts, etc. Do not drop empty
    dicts or lists.

    :param data: Object to process for None values.
        Returns a new dict if passed a dict;
        returns a new list if passed a list;
        returns the argument unchanged otherwise.
    """
    if isinstance(data, dict):
        new_data = {}
        for k, v in data.items():
            v = drop_none_values(v)
            if v is not None:
                new_data[k] = v
    elif isinstance(data, list):
        new_data = []
        for v in data:
            v = drop_none_values(v)
            if v is not None:
                new_data.append(v)
    else:
        new_data = data
    return new_data

Here's my test case, that function above is in a file/module named util:

def test_drop_none_values():
    data = {
        'first': None,
        'second': None,
        'empty': [],
        'alist': ['string', None, {'foo': None, 'bar': 'baz'}]
    }
    assert len(data) == 4
    stripped = util.drop_none_values(data)
    assert len(stripped) == 2
    assert len(stripped['alist']) == 2
    assert len(stripped['alist'][1]) == 1

Remove empty dicts in nested dictionary with recursive function

4 Answers4

Linked