54

Why does the json serialization not work for datetime objects . As I understand json serialization the basic idea for any object can be call the __str__ builtin function and then urlencode the object that you get as a response. But in case of datetime i get the following error 

TypeError: datetime.datetime(2012, 5, 23, 18, 38, 23, 37566) is not JSON serializable

while there is a __str__ i.e a way of stringifying the object already available , But it seems like a conscious decision to not do it , why would that be the case?

dusual
  • 2,097
  • 3
  • 19
  • 26
  • 9
    You're mis-understanding how json works. It has nothing to do with the __str__ method. JSON doesn't have a date-time type, period, so it is impossible to losslessly encode a date-time in JSON, without some sort of special logic on the receiving end. Thus the library (logically) makes you do it yourself by converting to a Unix timestamp or ISO date string or something and making it explicit that conversion is necessary. – Tyler Eaves May 23 '12 at 13:56
  • 3
    @TylerEaves This is anything but logical. It is possible to losslessly encode a datetime to a string or int, and many use cases that call for converting from dict, to json, to dict again never leaving the python ecosystem, yet the json module can't handle this case without a custom datetime handler? Really?! Looking at the large number of stackoverflow questions on the topic, I'd say I'm not alone in my incredulity. – Joshua Kolden Apr 26 '15 at 21:37

4 Answers4

92

No it doesn't work that way in json module. The module provides you with a default encoder: json.JSONEncoder. You need to extend this to provide your implementation of default method to serialize objects. Something like this:

import json
import datetime
from time import mktime

class MyEncoder(json.JSONEncoder):

    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return int(mktime(obj.timetuple()))

        return json.JSONEncoder.default(self, obj)

print json.dumps(obj, cls=MyEncoder)

As others correctly pointed out, the reason is that the standard for json does not specify how date time can be represented.

frlan
  • 6,950
  • 3
  • 31
  • 72
Vikas
  • 8,790
  • 4
  • 38
  • 48
  • 4
    how to decode this datetime at the front end? i am asking about the decoding of this decimal. – A.J. Feb 20 '14 at 06:57
  • 1
    @Clayton, what you'll get in JSON response is unix timestamp. [This question](http://stackoverflow.com/questions/847185/convert-a-unix-timestamp-to-time-in-javascript) answers how to convert unix timestamp to date. – Vikas Oct 23 '14 at 19:13
  • 3
    @Vikas Thanks. I followed your code but I slightly modified the `isinstance` check part to `if isinstance (obj, datetime.datetime): return obj.strftime ('%Y/%m/%d/%H/%M/%S') elif isinstance (obj, datetime.date): return obj.strftime ('%Y/%m/%d')`. – gsinha Feb 22 '15 at 15:29
  • Well using mktime does work I found what I feel is a better way. JSON uses ISO 8601 formatting for its time stamps. To get the same kind of time formatting out of python all you need to do is take your datetime object and add ```.isoformat()``` to the end. – SudoKid Jun 24 '16 at 19:04
10

How would you like them to be serialized?

JSON doesn't specify how to handle dates, so the python json library cannot make the decision on how to then represent these for you. That completely depends on how the other side (browser, script, whatever) handles dates in JSON as well.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • 3
    With a default serialization one doesn't necessarily care. You just need to be able to get into the json format and back out without too much trouble. Certainly failing on datetime entirely is guaranteed to be the wrong answer. – Joshua Kolden Apr 26 '15 at 21:43
  • @JoshuaKolden: but in an *interchange format* you do need to care. How does the JSON library in Java or in a JavaScript app want the date? – Martijn Pieters Apr 26 '15 at 21:44
  • 2
    Then that is when one would need to customize. Out of the box if I'm just going from python to json to python, not working by default on common data types just adds unnecessary overhead. Plus those other languages have standard conversions to json as well, so one could simply start with that as a good default. – Joshua Kolden Apr 26 '15 at 21:47
  • 1
    No, there is no such default. The standard doesn't specify any. – Martijn Pieters Apr 26 '15 at 22:17
  • 1
    And that prevents Python from encoding datetimes why? Other languages have no such problem. Json's data abstraction makes it *more* type flexible, not less so. Most languages choose (for themselves) a default datetime string encoding, and then have at least the ability to read and write datetime values to json consistently within the language. Python forces incompatibility not only with external systems, but even two different Python apps may handle datetime/json differently. – Joshua Kolden Apr 26 '15 at 22:41
  • 1
    But there is no interoperability here. [Python's philosophy](https://www.python.org/dev/peps/pep-0020/) include the phrases *refuse the temptation to guess* and *explicit is better than implicit*. Without a standard there is no sensible default, Python refuses to guess here and forces you to explicitly pick one instead. The majority of usecases for JSON is in communicating with other *unknown* clients that even if they had a default datetime serialisation all have *a different idea about the format*. – Martijn Pieters Apr 26 '15 at 22:48
  • 2
    It is not hard to pick a format, but that is not all that one must do. As an example it is non-trivial to get sqlalchemy to work with dictionaries (containing datetimes at various nesting levels) stuffed into postgresql JSON columns. A trivial task in go, ruby, and javascript. This is just dictionary/hash serialization after all, yet python can't do it. Python *does* make a choice in terms of what format is acceptable for creating datetime from strings, and there are also general standards, so the 'python philosophy' seems at best circular logic in this case. – Joshua Kolden Apr 26 '15 at 23:09
  • @JoshuaKolden: if json.dumps would encode datetimes automatically, then json.loads would need to decode them automatically. So JSON strings are always Python strings, unless they happen to be formatted in some way that json.loads decides is a datetime, then they become datetimes. Even if they were just strings in the first place. – RemcoGerlich Jul 26 '16 at 12:50
  • @RemcoGerlich Precisely so. This is how JSON works. Context defines the meaning of the 6 JSON types. Without context a string is just a string, with context you know what type you're expecting. In JSON having functional symmetry is unimportant, being functional is. – Joshua Kolden Jul 26 '16 at 20:54
8

A simple way to patch the json module such that serialization would support datetime. 

import json
import datetime

json.JSONEncoder.default = lambda self,obj: (obj.isoformat() if isinstance(obj, datetime.datetime) else None)

Than use json serialization as you always do - this time with datetime being serialized as isoformat.

json.dumps({'created':datetime.datetime.now()})

Resulting in: '{"created": "2015-08-26T14:21:31.853855"}'

See more details and some words of caution at: StackOverflow: JSON datetime between Python and JavaScript

Community
  • 1
  • 1
davidhadas
  • 2,333
  • 21
  • 17
2

If you want to get encoding and decoding of datetimes without having to implement it, you can use json_tricks, which is a wrapper that adds encoding and decoding for various popular types. Just install:

pip install json_tricks

and then import from json_tricks instead of json, e.g.:

from json_tricks import dumps, loads
json = dumps({'name': 'MyName', 'birthday': datetime.datetime(1992, 5, 23, 18, 38, 23, 37566)})
me = loads(json)

Disclaimer: it's made by me. Because I had the same problem.

If you want to automatically serialize anything that can be stringified, you can do that with just the standard implementation very easily:

dumps(obj, default=str)

But note that this has disadvantages, e.g. none of it will be deserialized without extra effort, and maybe sometimes you just don't want to serialize something (like a function of a big numpy array) but get a warning instead, which this method will silence.

Mark
  • 18,730
  • 7
  • 107
  • 130