How to write and read datetime dictionaries

Question

I need an efficient way to write files containing dictionaries including datetime, and then be able to read them as dicts. Dicts like these:

my_dict = {'1.0': [datetime.datetime(2000, 1, 1, 0, 0, 0, 000000, tzinfo=tzutc())], '2.0': [datetime.datetime(2000, 1, 1, 0, 0, 0, 000000, tzinfo=tzutc())]}

Trying to dump with json:

with open("my_file.json", 'w+') as f:
    json.dump(my_dict, f)

TypeError: Object of type 'datetime' is not JSON serializable

Also tried writing the entire dict as a String and then importing it with yaml, which almost worked, but got the indexing messed up.

with open("my_file", 'w+') as f:
    f.write(str(my_dict))

with open("my_file", 'r') as f:
    s = f.read()
    new_dict = yaml.load(s)

print(new_dict['1.0'][0])

Output:   datetime.datetime(2000
Expected: 2000-01-01 00:00:00+00:00

score 5 · Answer 1 · answered Jun 19 '17 at 13:26

5

You might feel it's heavy handed but the proper way in python is to use a custom encoder. You can choose your formatting (iso here but I'd suggest using datetime.date() in your example.

from datetime import date,datetime
import json, re

class DateTimeAwareEncoder(json.JSONEncoder):
    def default(self, o):
        if isinstance(o, datetime):
            return o.isoformat()

        return json.JSONEncoder.default(self, o)

json.dumps(yourobj, cls=DateTimeAwareEncoder)

How to use a custom decoder is much more complicated and usually best handled by loading with json.loads then transforming the known keys to avoid the custom decoder.

answered Jun 19 '17 at 13:26

Phil Cooper

5,747
1
25
41

Thank you for this. – Aminoff Jun 19 '17 at 13:41
You, sir, saved my day. – Sander Vanden Hautte Feb 22 '18 at 15:37
This works well, thanks for this – Pierre Apr 01 '20 at 14:11

score 1 · Accepted Answer · answered Jun 19 '17 at 12:12

You can use Pickle. It is used for serializing and de-serializing a Python object structure. Any object in python can be pickled so that it can be saved on disk. What pickle does is that it “serialises” the object first before writing it to file. Pickling is a way to convert a python object (list, dict, etc.) into a character stream. The idea is that this character stream contains all the information necessary to reconstruct the object in another python script. Source

This would work:

>>> import pickle
>>> file_Name = "testfile"
>>> fileObject = open(file_Name,'wb')
>>> my_dict = {'1.0': [datetime.datetime(2000, 1, 1, 0, 0, 0, 000000, tzinfo=tzutc())], '2.0': [datetime.datetime(2000,
1, 1, 0, 0, 0, 000000, tzinfo=tzutc())]}
>>> pickle.dump(my_dict, fileObject)
>>> fileObject.close()
>>> fileObject = open(file_Name,'r')
>>> my_dict_loaded = pickle.load(fileObject)
>>> my_dict_loaded
{'1.0': [datetime.datetime(2000, 1, 1, 0, 0, 0, 000000, tzinfo=tzutc())], '2.0': [datetime.datetime(2000,
1, 1, 0, 0, 0, 000000, tzinfo=tzutc())]}

I liked this approach, was alot faster than I expected. – Aminoff Jun 19 '17 at 13:42 — Aminoff, Jun 19 '17 at 13:42

score 1 · Answer 3 · answered Jun 19 '17 at 12:20

You can use the strftime method:

my_dict = {
    '1.0': [
               datetime.datetime(
                   2000, 
                   1, 
                   1, 
                   0, 
                   0, 
                   0, 
                   000000, 
                   tzinfo=tzutc()
              ).strftime("%Y-%m-%d %H:%M:%S")],
    '2.0': [
              datetime.datetime(
                   2000, 
                   1, 
                   1, 
                   0, 
                   0, 
                   0, 
                   000000, 
                   tzinfo=tzutc()
              ).strftime("%Y-%m-%d %H:%M:%S")
           ]

}

How to write and read datetime dictionaries

3 Answers3