a way to update csv file without rewriting

Question

import pandas as pd
import csv,sched,nltk,arrow
from time import perf_counter


def ngram_search():
    #code here
    item['length']=len(word_tokenize(s)) # number
    item['time edit']=arrow.utcnow().shift(hours=-1).humanize().upper() # text
    item['top1'] = "word"
    return item
path=r"C:\Users\Sublime Text\products.csv"
df = pd.read_csv(path)
saved_column = df['Name of Product']
for i in saved_column:
    name=i.replace(",","+").replace(";","+")
    item=ngram_search(name)
    #update csv file here add the columns of items to the csv file

I am trying to add to an existing csv file (15 columns, 283216 row), the function returns a dictionary of 3 values. Is there a way to directly update the csv file without rewriting it?

Maybe pandas.DataFrame.append? `https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.append.html` — Mark Moretto, Apr 21 '19 at 01:10
you can use `to_csv(filename, mode="a")` to append to existing file. But you have to write the same number of columns and in the same order. — furas, Apr 21 '19 at 01:32
Possible duplicate of [How to add pandas data to an existing csv file?](https://stackoverflow.com/questions/17530542/how-to-add-pandas-data-to-an-existing-csv-file) — Craig, Apr 21 '19 at 02:16

score 1 · Answer 1 · edited Jul 04 '21 at 08:41

DataFrame.to_csv() has option mode="a" to append to existing file.

You have to only write the same number of columns and write them in the same order (with the same separator, etc.) to create correctly formatted CVS.

This way you can only append to the end of the file. If you need to write between existing rows or add a new column then you have to read all. OR you have to read from one file and write to a new one (all data with changes).

a way to update csv file without rewriting

1 Answers1