10

I'm seemingly unable to deserialize my MongoDB JSON document with the BSON json_util.

The json.loads function is choking on the ObjectId() string. I had understood json_util capable of handling MongoDB's ObjectId format and transforming into usable JSON.

Python code:

import json    
from bson import json_util

s = "{u'_id': ObjectId('4ed559abf047050c58000000')}"
u = json.loads(s, object_hook=json_util.object_hook)

I get the decoder exception:

...
    u = json.loads(s, object_hook=json_util.object_hook)
  File "\python27\lib\json\__init__.py", line 339, in loads
    return cls(encoding=encoding, **kw).decode(s)
  File "\python27\lib\json\decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "\python27\lib\json\decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting property name: line 1 column 1 (char 1)

Am I missing something?

jdev
  • 729
  • 2
  • 13
  • 36

2 Answers2

20

I think your string form actually looks like the python representation...

s = '{"_id": {"$oid": "4edebd262ae5e93b41000000"}}'
u = json.loads(s, object_hook=json_util.object_hook)

print u  # Result:  {u'_id': ObjectId('4edebd262ae5e93b41000000')}

s = json.dumps(u, default=json_util.default)

print s  # Result:  {"_id": {"$oid": "4edebd262ae5e93b41000000"}}

The bson.json_util.object_hook function does not seem to have any type of handling for there being ObjectId() in the actual json string representation.

jdi
  • 90,542
  • 19
  • 167
  • 203
  • Ok, after looking at @dcrosta's suggestion to review MongoDB Extended JSON http://www.mongodb.org/display/DOCS/Mongo+Extended+JSON, let me instead try converting the data_oid from TenGen BSON type to Strict JSON, i.e. "$oid". – jdev Dec 07 '11 at 16:36
  • 1
    The json.dumps suggestion worked perfectly. s = json.dumps(u, default=json_util.default), output {"_id": {"$oid": "4ed559abf047050c58000000"}}. Thanks @jdi – jdev Dec 07 '11 at 17:47
  • Hi All, I also want to conver ObjectId() of the mongo result, i tried to execute above code, i am getting the following error `NameError: global name 'json_util' is not defined` is any library i need to import? , i have imported `bson` library but still same error – Sarada Akurathi Feb 08 '17 at 09:20
  • @SaradaAkurathi when I wrote this answer over. 5 years ago, json_util was part of the bson module (as documented in the answer). Is that no longer the case? – jdi Feb 08 '17 at 09:26
  • @jdi, i found this [json_util](http://api.mongodb.com/python/current/api/bson/json_util.html) for json_util. option `bson.json_util.default` is still there. problem earlier with wrong import, now i imported correct library, the above error is gone but still result is not converted. Can you help what went wrong – Sarada Akurathi Feb 08 '17 at 09:41
  • @SaradaAkurathi I am not sure what your issue is. You may be best served by opening a new question with the details of your problem. This question is quite old, and comments are not really the best place to troubleshoot – jdi Feb 08 '17 at 09:44
  • object_hook=json_util.object_hook does the magic. Thanks for sharing:) – Ashwini Aug 04 '21 at 08:07
4

There are two problems here:

  1. The string you're attempting to JSON-decode is not JSON, it's the string representation of a Python dictionary. In particular, the problem is that u'_id' is not a valid JSON key (JSON keys are quoted strings; the "u" here indicates a Python unicode string, which is meaningless in JSON)

  2. json_util.object_hook doesn't make ObjectId available to JSON; the json module will decode the JSON, and then call the object_hook callback with each decoded object. json_util.object_hook will look for certain patterns as defined in the strict mode of MongoDB Extended JSON.

See @jdi's answer for examples of how to properly use json_util.

dcrosta
  • 26,009
  • 8
  • 71
  • 83
  • RE: 1. Hi dcrosta, the string I had used in my example was taken from the app log, hence the extra python unicode 'u'. – jdev Dec 07 '11 at 16:28