3

I have an python dict whose keys and values are strings, integers and other dicts and tuples (json does not support those). I want to save it to a text file and then read it from the file. Basically, I want a read counterpart to the built-in print (like in Lisp).

Constraints:

  1. the file must be human readable (thus pickle is out)
  2. no need to detect circularities.

Is there anything better than json?

Community
  • 1
  • 1
sds
  • 58,617
  • 29
  • 161
  • 278
  • 4
    What would make something better than json in your estimation? – BrenBarn Jan 20 '15 at 21:36
  • @BrenBarn: `read` which matches `print` – sds Jan 20 '15 at 21:39
  • @sds what do you mean read is for input print is for output ... they are two fundamentally different things – Joran Beasley Jan 20 '15 at 21:46
  • cf [print](http://www.lispworks.com/documentation/HyperSpec/Body/f_wr_pr.htm) and [read](http://www.lispworks.com/documentation/HyperSpec/Body/f_rd_rd.htm) – sds Jan 20 '15 at 22:03
  • Honestly, json is your answer, and that's why it's taken over in the least 5 years. What legibility issues does json have? There are tons of json indenter and pretty-printer utilities - use them and it is human-readable. json is also extremely performant. – smci Jan 20 '15 at 22:14
  • @smci: yes, I am afraid I will have to stick with json – sds Jan 20 '15 at 22:16
  • @sds: honestly, it's briliant. Just find a nice indenter/pretty-printer you like, associate it with .json file type, and you'll never look back. There are tons of blazing fast and performant json libraries. The 'human-readibility' thingy is only a psychological hurdle. I had the same reaction as you when I first saw it. – smci Jan 20 '15 at 22:21

2 Answers2

9

You could use repr() on the dict, then read it back in and parse it with ast.literal_eval(). It's as human readable as Python itself is.

Example:

In [1]: import ast

In [2]: x = {}

In [3]: x['string key'] = 'string value'

In [4]: x[(42, 56)] = {'dict': 'value'}

In [5]: x[13] = ('tuple', 'value')

In [6]: repr(x)
Out[6]: "{(42, 56): {'dict': 'value'}, 'string key': 'string value', 13: ('tuple', 'value')}"

In [7]: with open('/tmp/test.py', 'w') as f: f.write(repr(x))

In [8]: with open('/tmp/test.py', 'r') as f: y = ast.literal_eval(f.read())

In [9]: y
Out[9]:
{13: ('tuple', 'value'),
 'string key': 'string value',
 (42, 56): {'dict': 'value'}}

In [10]: x == y
Out[10]: True

You may also consider using the pprint module for even friendlier formatted output.

user108471
  • 2,488
  • 3
  • 28
  • 41
  • 2
    its worth noting that `simplejson` is orders of magnitude faster than this although this is slightly faster than naive `json` – Joran Beasley Jan 20 '15 at 21:47
  • 2
    @JoranBeasley That's good to know. I can't think of much else that's as simple to implement as JSON and similarly human readable. – user108471 Jan 20 '15 at 21:50
  • 1
    Given json's inability to handle tuple keys, this seems to be the _only_ answer. Thanks. – sds Jan 26 '15 at 23:01
  • If the output of `repr()` is insufficiently pretty for you, you may also want to consider the pprint function from the pprint module, which is designed to pretty-print python objects in a more readable way. The output of `pprint()` can just as easily be parsed by `ast.literal_eval()`. – user108471 Jan 27 '15 at 15:23
1

Honestly, json is your answer [EDIT: so long as the keys are strings, didn't see the part about dicts as keys], and that's why it's taken over in the least 5 years. What legibility issues does json have? There are tons of json indenter, pretty-printer utilities, browser plug-ins [1][2] - use them and it certainly is human-readable. json(/simplejson) is also extremely performant (C implementations), and it scales, and can be processed serially, which cannot be said for the AST approach (why be eccentric and break scalability?).

This also seems to be the consensus from 100% of people answering you here... everyone can't be wrong ;-) XML is dead, good riddance.

  1. How can I pretty-print JSON? and countless others
  2. Browser JSON Plugins
Community
  • 1
  • 1
smci
  • 32,567
  • 20
  • 113
  • 146
  • sorry, ["json is your answer" is BS](http://stackoverflow.com/questions/7001606/json-serialize-a-dictionary-with-tuples-as-key) – sds Jan 26 '15 at 22:43
  • @sds: you need to explain why, you can't just post a swearword. For what it's worth, I experimented with other formats over a couple of years until I reluctantly came to the same conclusion. There are very solid reasons (faster parsing, performance, standardization, tons of third-party tools/ utilities/ libraries). – smci Jan 26 '15 at 22:56
  • @sds Also you need to articulate a better method: the OP's AST approach, or something else? – smci Jan 26 '15 at 22:57
  • The link clearly explains that json cannot serialize quite a lot of objects. – sds Jan 26 '15 at 22:58
  • I don't have a better method; if I did - I would not have asked here. However, json cannot be an answer. It might be that one could write a json-based serializer, but the simple "use `json.dump`" is definitely not an answer the serialization question. – sds Jan 26 '15 at 22:59
  • So: JSON only supports strings as keys, tuples/lists/dicts/sets/arbitrary objects not allowed. Would need to workaround. Depends on what the OP's dict keys are. See [Best way to encode tuples with json](http://stackoverflow.com/questions/715550/best-way-to-encode-tuples-with-json) – smci Jan 26 '15 at 23:01
  • Nope, it does not depend on my actual data. "Serialization" means universality. It means that any object which can be meaningfully saved and restored should be handled seamlessly (e.g., I am not asking for an ability to save/restore streams). – sds Jan 26 '15 at 23:05