PyMongo & Pandas - Update multiple records in MongoDB collection from dataframe by matching id

Question

Is it possible to update multiple records in MongoDB collection from dataframe by matching id?

dataframe

    _id                                      text                            sentiment
0   5ec299fa905e038dec3c8e93    Kederi Yusof· basikal salah Najib.Tayar pa...   1
1   5ec49452bfcd4786382fe21f    Serindik.com·2 laaa mimpi UMNO.... XPM7 pul...  0
2   5ec40e8d28fb32986041df16    Newpaper24·4m1MDB: Najib Razak’s court, accuse...   -1
3   5ec44c0b255995f0522fe1ec    falseprophet· low pesuruh najib. Budak des...   1
4   5ed2ab347d23a5d56d59a730    Kamaluddin 阿列克斯 தீன்·11m-anak-najib-dalam-sena...   0

code

updates = []
for document in db.twitter.find():
    for index, row in document.iterrows():
        if(row['_id']==a['_id']):
            updates.append(UpdateOne({'_id': row['_id']}, {'$set': {'sentiment': row['sentiment']}}, upsert=True))
            break

db.twitter.bulk_write(updates)

But I got AttributeError: 'dict' object has no attribute 'iterrows' error message

Check this out: https://stackoverflow.com/questions/13710770/how-to-update-values-using-pymongo — Mayank Porwal, Jun 02 '20 at 19:18
I don't think, that you need second loop. `document` is a PyMongo object, it's not a DataFrame object. — ngShravil.py, Jun 02 '20 at 19:22

score 3 · Accepted Answer · answered Jun 02 '20 at 20:56

You don't need the find loop, just use iterrows() to get the data and UpdateOne to perform the upsert.

from pymongo import MongoClient, UpdateOne
import pandas as pd

db = MongoClient()['mydatabase']

data = [['ec299fa905e038dec3c8e93', 'Kederi Yusof· basikal salah Najib.Tayar pa...', 1],
        ['ec49452bfcd4786382fe21f', 'Serindik.com·2 laaa mimpi UMNO.... XPM7 pul...', 0]]

df = pd.DataFrame(data, columns=['_id', 'text', 'sentiment'])

updates = []

for _, row in df.iterrows():
    updates.append(UpdateOne({'_id': row.get('_id')}, {'$set': {'sentiment': row.get('sentiment')}}, upsert=True))

db.twitter.bulk_write(updates)

Do you have any alternative to `iterrows()` as it is very slow operation for a large dataframe (Let's say 1 million rows). — BetterCallMe, Dec 28 '22 at 15:50

PyMongo & Pandas - Update multiple records in MongoDB collection from dataframe by matching id

1 Answers1

Linked