JSON dumps Decimal is not serializable

Question

Edit: Added new output with

logging.info(id(type(Decimal(10))))
logging.info(id(type(obj)))

I am trying to dump to JSON. I know that I need to cast Decimal as a float so here is my code:

def get(self):
    results = CheckPulseRequest.get(get_args(self.request))
    self.response.headers['Content-Type'] = 'application/json'
    self.response.out.write(json.dumps(results, cls=DateTimeEncoder))

And then I have defined DateTimeEncoder as follows:

class DateTimeEncoder(json.JSONEncoder):
    logging.info("In DateTimeEncoder Class")

    def default(self, obj):
        logging.info(id(type(Decimal(10))))
        logging.info(id(type(obj)))
        logging.info(type(Decimal(10)))
        logging.info(type(obj))
        logging.info(isinstance(Decimal(10), (int, long, float, complex, Decimal)))
        logging.info(isinstance(obj, (int, long, float, complex, Decimal)))

        if isinstance(obj, Decimal):
            logging.info("Is Instance of Decimal")
            return float(obj)
        elif hasattr(obj, 'isoformat'):
            logging.info("Is ISO Format")
            return obj.isoformat()
        else:
            logging.info("Else")
            return json.JSONEncoder.default(self, obj)

And I get the following log output. My object that is a Decimal doesn't get converted to a float. Notice that the object has the same type as a Decimal but the isinstance test only is True on the Decimal(10) test. What am I missing?

INFO     2015-02-23 18:49:45,737 check_pulse_request.py:25] Getting Check Pulse Request ID: 4
INFO     2015-02-23 18:49:46,374 main.py:179] 140647859901984
INFO     2015-02-23 18:49:46,375 main.py:180] 140647856664608
INFO     2015-02-23 18:49:46,375 main.py:181] <class 'decimal.Decimal'>
INFO     2015-02-23 18:49:46,375 main.py:182] <class 'decimal.Decimal'>
INFO     2015-02-23 18:49:46,375 main.py:183] True
INFO     2015-02-23 18:49:46,375 main.py:184] False
INFO     2015-02-23 18:49:46,376 main.py:193] Else
ERROR    2015-02-23 18:49:46,376 webapp2.py:1552] Decimal('-122.39906660000000') is not JSON serializable
Traceback (most recent call last):
  File "/Users/sheridangray/Projects/adt-bundle-mac-x86_64-20140702/sdk/google-cloud-sdk/platform/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/Users/sheridangray/Projects/adt-bundle-mac-x86_64-20140702/sdk/google-cloud-sdk/platform/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/Users/sheridangray/Projects/adt-bundle-mac-x86_64-20140702/sdk/google-cloud-sdk/platform/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/Users/sheridangray/Projects/adt-bundle-mac-x86_64-20140702/sdk/google-cloud-sdk/platform/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/Users/sheridangray/Projects/adt-bundle-mac-x86_64-20140702/sdk/google-cloud-sdk/platform/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/Users/sheridangray/Projects/adt-bundle-mac-x86_64-20140702/sdk/google-cloud-sdk/platform/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/Users/sheridangray/Projects/city-pulse-appengine/city-pulse/main.py", line 38, in get
    self.response.out.write(json.dumps(results, cls=DateTimeEncoder))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 250, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/Users/sheridangray/Projects/city-pulse-appengine/city-pulse/main.py", line 194, in default
    return json.JSONEncoder.default(self, obj)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: Decimal('-122.39906660000000') is not JSON serializable

You don't have any other classes named `Decimal` in your code, by any chance? — Martijn Pieters, Feb 23 '15 at 08:42
Hrm, not sure why this was downvoted. There is code, there is an error message, what's not to like? Sure, the error looks incongruous, but clearly there is a type mismatch here as `type(obj)` makes it *look* like it is a `decimal.Decimal()` instance. I'm just not sure `Decimal` in the `default` code is the same class as what was being passed in. Two different `decimal.Decimal` classes would explain the error. — Martijn Pieters, Feb 23 '15 at 08:46
Can you add logging for `id(Decimal)` and `id(type(obj))` to your code and re-run? — Martijn Pieters, Feb 23 '15 at 08:47
I would be careful serialising decimals as floats as you may lose precision -- as floats only have a finite set of values in comparison to Decimal which has an infinite set of values. This only really matters if you have values with 20 significant figures or more. Consider using [`simplejson`](https://pypi.python.org/pypi/simplejson/) using `simplejson.dumps(obj, use_decimal=True)`. — Dunes, Feb 23 '15 at 10:11
@Dunes: all `simplejson` does is use `str(Decimal)`. The OP could simply do the same here. — Martijn Pieters, Feb 23 '15 at 14:43
@MartijnPieters That wouldn't produce entirely the same result. `simplejson.dumps(Decimal("1.1")` produces `'1.1'` -- a number, whereas `json.dumps(Decimal("1.1"), cls=DecimalEncoder)` produces `'"1.1"'` -- a string. In addition simplejson is also able to parse all floats as Decimal with no fuss when de-serialising (if that is needed). — Dunes, Feb 23 '15 at 14:52
@Dunes: ah, yes, I see what you mean there. Note that `json.load[s]()` takes a `parse_float` hook to load floating point literals; you can set that to `decimal.Decimal`. — Martijn Pieters, Feb 23 '15 at 14:58

score 0 · Answer 1 · answered Feb 24 '15 at 00:12

0

I didn't really "fix" the issue but worked around it by changing my MySQL database columns to be a FLOAT.

answered Feb 24 '15 at 00:12

Sheridan Gray

698
1
9
23

If precision matters (e.g. for currency, etc.) you may not want to use `float`. – tzaman Feb 24 '15 at 00:17
Precision isn't terribly important in this case as it's an approximation in the first place :) – Sheridan Gray Feb 24 '15 at 23:53

JSON dumps Decimal is not serializable

1 Answers1