Pymongo: Best way to remove $oid in Response

Question

I have started using Pymongo recently and now I want to find the best way to remove $oid in Response

When I use find:

result = db.nodes.find_one({ "name": "Archer" }

And get the response:

json.loads(dumps(result))

The result would be:

{
  "_id": {
  "$oid": "5e7511c45cb29ef48b8cfcff"
  },
  "about": "A jazz pianist falls for an aspiring actress in Los Angeles."
}

My expected:

{
  "_id": "5e7511c45cb29ef48b8cfcff",
  "about": "A jazz pianist falls for an aspiring actress in Los Angeles."
}

As you seen, we can use:

resp = json.loads(dumps(result))
resp['id'] = resp['id']['$oid']

But I think this is not the best way. Hope you guys have better solution.

score 2 · Accepted Answer · answered Mar 21 '20 at 08:13

2

You can take advantage of aggregation:

result = db.nodes.aggregate([{'$match': {"name": "Archer"}}
                             {'$addFields': {"Id": '$_id.oid'}},
                             {'$project': {'_id': 0}}])
data = json.dumps(list(result))

Here, with $addFields I add a new field Id in which I introduce the value of oid. Then I make a projection where I eliminate the _id field of the result. After, as I get a cursor, I turn it into a list.

It may not work as you hope but the general idea is there.

answered Mar 21 '20 at 08:13

Tobin

2,029
2
11
19

When I use you code, the `id` field completely disappear :( – KitKit Mar 21 '20 at 08:47
What if you do this: `{'$addFields': {"Id": '$_id.$oid'}}` ? – Tobin Mar 21 '20 at 08:54
Also disappear, bro. Remember I am using `Pymongo` – KitKit Mar 21 '20 at 10:05

score 1 · Answer 2 · answered Mar 21 '20 at 11:10

1

First of all, there's no $oid in the response. What you are seeing is the python driver represent the _id field as an ObjectId instance, and then the dumps() method represent the the ObjectId field as a string format. the $oid bit is just to let you know the field is an ObjectId should you need to use for some purpose later.

The next part of the answer depends on what exactly you are trying to achieve. Almost certainly you can acheive it using the result object without converting it to JSON.

If you just want to get rid of it altogether, you can do :

result = db.nodes.find_one({ "name": "Archer" }, {'_id': 0})
print(result)

which gives:

{"name": "Archer"}

answered Mar 21 '20 at 11:10

Belly Buster

8,224
2
7
20

Thanks Belly for the instruction. Exactly I don't want to get rid of the `_id` field like you said. I want to find the best way to map the `$oid` value to `_id` field in the response. So how can I get the expected result? – KitKit Mar 21 '20 at 11:22
I'll come back to my point ... why do you want to do this? To make it look pretty? What purpose is there to have it as a bit of JSON rather than have it as a nice result object primed to do whatever you need? – Belly Buster Mar 21 '20 at 11:51
I just simply want the json look prettier for frontend team. I wonder if is it good? – KitKit Mar 22 '20 at 06:59
`_id`s aren't the prettiest of things anyway. If that is your aim, then your original approach is as good as any.. – Belly Buster Mar 22 '20 at 12:43
I got it, maybe I will get rid of `_id` because the unnecessariness. Thank you – KitKit Mar 24 '20 at 08:01

score 1 · Answer 3 · answered Jul 30 '21 at 11:49

I am using some form of custom handler. I managed to remove $oid and replace it with just the id string:

# Custom Handler
def my_handler(x):
    if isinstance(x, datetime.datetime):
        return x.isoformat()
    elif isinstance(x, bson.objectid.ObjectId):
        return str(x)
    else:
        raise TypeError(x)

# parsing
def parse_json(data):
    return json.loads(json.dumps(data, default=my_handler))

result = db.nodes.aggregate([{'$match': {"name": "Archer"}}
                             {'$addFields': {"_id": '$_id'}},
                             {'$project': {'_id': 0}}])
data = parse_json(result)

score 1 · Answer 4 · answered Nov 21 '21 at 15:18

import re

def remove_oid(string):
    while True:
        pattern = re.compile('{\s*"\$oid":\s*(\"[a-z0-9]{1,}\")\s*}')
        match = re.search(pattern, string)
        if match:
            string = string.replace(match.group(0), match.group(1))
        else:
            return string

string = json_dumps(mongo_query_result)
string = remove_oid(string)

score 0 · Answer 5 · answered Aug 08 '22 at 18:28

In the second argument of find_one, you can define which fields to exclude, in the following way:

site_information = mongo.db.sites.find_one({'username': username}, {'_id': False})

This statement will exclude the '_id' field from being selected from the returned documents.

Mritunjay Paswan · Answer 6 · 2023-08-14T04:48:21.267

Pass your result to below function. (Tested at 5 level of nesting with $oid and $data)

def remove_special_fields(d):
    if isinstance(d, dict):
        new_dict = {}
        for key, value in d.items():
            if isinstance(value, dict) and ('$oid' in value or '$date' in value):
                new_dict[key] = value.get('$oid', value.get('$date'))
            else:
                new_dict[key] = remove_special_fields(value)
        return new_dict
    elif isinstance(d, list):
        return [remove_special_fields(item) for item in d]
    else:
        return d

Pymongo: Best way to remove $oid in Response

6 Answers6