Why are some numpy datatypes JSON serializable and others not?

Question

Numpy has a lot of different basic types, all of which are listed here.

I've tracked down an issue in my program to float32s not being JSON-serializable, so I've started testing all datatypes from the list above:

>>> import numpy as np
>>> from json import dumps
>>> dumps(np.bool(True))
'true'
>>> dumps(np.bool_(True))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/json/__init__.py", line 230, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.4/json/encoder.py", line 192, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.4/json/encoder.py", line 250, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.4/json/encoder.py", line 173, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: True is not JSON serializable
>>> dumps(np.int(0))
'0'
>>> dumps(np.int_(0))
Traceback (most recent call last):
  [...]
TypeError: 0 is not JSON serializable
>>> dumps(np.intc(0))
Traceback (most recent call last):
  [...]
TypeError: 0 is not JSON serializable
>>> dumps(np.intp(0))
Traceback (most recent call last):
  [...]
TypeError: 0 is not JSON serializable
>>> dumps(np.int8(0))
Traceback (most recent call last):
  [...]
TypeError: 0 is not JSON serializable
>>> dumps(np.int16(0))
Traceback (most recent call last):
  [...]
TypeError: 0 is not JSON serializable
>>> dumps(np.int32(0))
Traceback (most recent call last):
  [...]
TypeError: 0 is not JSON serializable
>>> dumps(np.int64(0))
Traceback (most recent call last):
  [...]
TypeError: 0 is not JSON serializable
>>> dumps(np.uint8(0))
Traceback (most recent call last):
  [...]
TypeError: 0 is not JSON serializable
>>> dumps(np.uint16(0))
Traceback (most recent call last):
  [...]
TypeError: 0 is not JSON serializable
>>> dumps(np.uint32(0))
Traceback (most recent call last):
  [...]
TypeError: 0 is not JSON serializable
>>> dumps(np.uint64(0))
Traceback (most recent call last):
  [...]
TypeError: 0 is not JSON serializable
>>> dumps(np.float(0))
'0.0'
>>> dumps(np.float_(0))
'0.0'
>>> dumps(np.float16(0))
Traceback (most recent call last):
  [...]
TypeError: 0.0 is not JSON serializable
>>> dumps(np.float32(0))
Traceback (most recent call last):
  [...]
TypeError: 0.0 is not JSON serializable
>>> dumps(np.float64(0))
'0.0'
>>> dumps(np.complex(0))
Traceback (most recent call last):
  [...]
TypeError: 0j is not JSON serializable
>>> dumps(np.complex_(0))
Traceback (most recent call last):
  [...]
TypeError: 0j is not JSON serializable
>>> dumps(np.complex64(0))
Traceback (most recent call last):
  [...]
TypeError: 0j is not JSON serializable
>>> dumps(np.complex128(0))
Traceback (most recent call last):
  [...]
TypeError: 0j is not JSON serializable

So, no complex type is serializable, that makes sense.

But bool works and bool_ doesn't. int works, but anything else with int in its name doesn't. float, float_ and float64 are all fine, but float16 and float32 are not.

Why is this the case? Obviously, they can all easily be converted to a string, the stacktrace even shows repr() being used to display their value. Might this be unintentional? Or is there a good reason for this behaviour?

Are you creating the `np.float32(...)` object directly? Or is it extracted from an array? Have tried serializing the `arr.tolist()` equivalent? — hpaulj, Jun 09 '17 at 19:44
If you need to serialize an element of an array, consider using the `.item()` method to convert it to a Python scalar. — hpaulj, Jun 09 '17 at 20:07
See also https://stackoverflow.com/questions/27050108/convert-numpy-type-to-python `Convert numpy type to python` — hpaulj, Jun 09 '17 at 20:18

Mike Müller · Accepted Answer · 2017-06-09T15:01:20.890

8

The data types that are JSON serializable are all Python built-ins:

>>> np.int is int
True
>>> np.float is float
True
>>> np.bool is bool
True

So all NumPy data types that you show are not JSON serializable. At least is consistent.

np.float_ is the same as np.float64 (tested on MacOS):

>>> np.float_ is np.float64
True

The help says:

np.float64

64-bit floating-point number. Character code 'd'. Python float compatible.

Whereas:

np.float32

32-bit floating-point number. Character code 'f'. C float compatible.

So, Python float compatible types work with json.dumps(), but C compatible ones don't.

edited Jun 09 '17 at 15:01

answered Jun 09 '17 at 13:47

Mike Müller

82,630
20
166
161

Then why do `float64` and `float_` work? `>>> np.float64 is float` and `>>> np.float_ is float` both return `False`. – iFreilicht Jun 09 '17 at 13:51
1

You can also look at the `__mro__`. `np.float64.__mro__` includes `float`, np.float32.__mro__` does not. – hpaulj Jun 09 '17 at 20:06

Why are some numpy datatypes JSON serializable and others not?

1 Answers1