How to convert string int JSON into real int with json.loads

Question

I'm trying to convert a string which represents a JSON object to a real JSON object using json.loads but it doesn't convert the integers:

(in the initial string, integers are always strings)

$> python
Python 2.7.9 (default, Aug 29 2016, 16:00:38)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> c = '{"value": "42"}'
>>> json_object = json.loads(c, parse_int=int)
>>> json_object
{u'value': u'42'}
>>> json_object['value']
u'42'
>>>

Instead of {u'value': u'42'} I'd like it becomes {u'value': 42}. I know I can run through the whole object, but I don't want to do that, it's not really efficient to do it manually, since this parse_int argument exists (https://docs.python.org/2/library/json.html#json.loads).

Thanks to Pierce's proposition:

Python 2.7.9 (default, Aug 29 2016, 16:00:38)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>>
>>> class Decoder(json.JSONDecoder):
...     def decode(self, s):
...         result = super(Decoder, self).decode(s)
...         return self._decode(result)
...     def _decode(self, o):
...         if isinstance(o, str) or isinstance(o, unicode):
...             try:
...                 return int(o)
...             except ValueError:
...                 try:
...                     return float(o)
...                 except ValueError:
...                     return o
...         elif isinstance(o, dict):
...             return {k: self._decode(v) for k, v in o.items()}
...         elif isinstance(o, list):
...             return [self._decode(v) for v in o]
...         else:
...             return o
...
>>>
>>> c = '{"value": "42", "test": "lolol", "abc": "43.4",  "dcf": 12, "xdf": 12.4}'
>>> json.loads(c, cls=Decoder)
{u'test': u'lolol', u'dcf': 12, u'abc': 43.4, u'value': 42, u'xdf': 12.4}

Answer 1

In addition to the Pierce response, I think you can use the json.loads object_hook parameter instead of cls one, so you don't need to walk the json object twice.

For example:

def _decode(o):
    # Note the "unicode" part is only for python2
    if isinstance(o, str) or isinstance(o, unicode):
        try:
            return int(o)
        except ValueError:
            return o
    elif isinstance(o, dict):
        return {k: _decode(v) for k, v in o.items()}
    elif isinstance(o, list):
        return [_decode(v) for v in o]
    else:
        return o

# Then you can do:
json.loads(c, object_hook=_decode)

As @ZhanwenChen pointed out in a comment, the code above is for python2. For python3 you'll need to remove the or isinstance(o, unicode) part in the first if condition.

Answer 2

As we established in the comments, there is no existing functionality to do this for you. And I read through the documentation and some examples on the JSONDecoder and it also appears to not do what you want without processing the data twice.

The best option, then, is something like this:

class Decoder(json.JSONDecoder):
    def decode(self, s):
        result = super().decode(s)  # result = super(Decoder, self).decode(s) for Python 2.x
        return self._decode(result)

    def _decode(self, o):
        if isinstance(o, str) or isinstance(o, unicode):
            try:
                return int(o)
            except ValueError:
                return o
        elif isinstance(o, dict):
            return {k: self._decode(v) for k, v in o.items()}
        elif isinstance(o, list):
            return [self._decode(v) for v in o]
        else:
            return o

This has the downside of processing the JSON object twice — once in the super().decode(s) call, and again to recurse through the entire structure to fix things. Also note that this will convert anything which looks like an integer into an int. Be sure to account for this appropriately.

To use it, you do e.g.:

>>> c = '{"value": "42"}'
>>> json.loads(c, cls=Decoder)
{'value': 42}

Answer 3

For my solution I used object_hook, which is useful when you have nested json

>>> import json
>>> json_data = '{"1": "one", "2": {"-3": "minus three", "4": "four"}}'
>>> py_dict = json.loads(json_data, object_hook=lambda d: {int(k) if k.lstrip('-').isdigit() else k: v for k, v in d.items()})

>>> py_dict
{1: 'one', 2: {-3: 'minus three', 4: 'four'}}

There is a filter only for parsing a json key to int. You can use int(v) if v.lstrip('-').isdigit() else v to filter for json values too.

Answer 4

In addition to @juanra and therefore @Pierce Darragh I added a conversion for boolean values from string. My example is a dict converted from XML that contains 'true' and 'false' that won't be loaded as JSON-boolean True and False automatically with the suggested answers.

def _decode(o):
    if isinstance(o, str):
        if o.lower() == 'true':
            return True
        elif o.lower() == 'false':
            return False
        else:
            try:
                return int(o)
            except ValueError:
                return o
    elif isinstance(o, dict):
        return {k: _decode(v) for k, v in o.items()}
    elif isinstance(o, list):
        return [_decode(v) for v in o]
    else:
        return o

According what you need you can also include other strings for boolean conversion with Converting from a string to boolean in Python?

Answer 5

def convert_to_int(params):
    for key in params.keys():
        if isinstance(params[key], dict):
            convert_to_int(params[key])
        elif isinstance(params[key], list):
            for item in params[key]:
                if not isinstance(item, (dict, list)):
                    item = int(item)
                else:
                    convert_to_int(item)
        else:
            params[key] = int(params[key])
    return params


print convert_to_int({'a': '3', 'b': {'c': '4', 'd': {'e': 5}, 'f': [{'g': '6'}]}})