Python Pandas - Write New CSV Header Row without Reading/ReWriting Entire File

Question

I have a 27GB CSV file and I want to simply rename the header rows. Can I do this without reading the entire file into a dataframe and then writing the entire file again?

This is essentially what I want to do, but without re-writing the whole 27GB file.

data = pd.read_csv(filename,sep="|",nrows=2)
data.head()

LOC_ID  UPC FW  BOP_U   BOP_$
0   17  438531560821    201712  1   40.0
1   239 438550152328    201719  2   28.8


data.columns = ['WHSE','SKU','PERIOD','QUANTITYONHAND','DOLLARSONHAND']
data.head()


   WHSE           SKU  PERIOD  QUANTITYONHAND  DOLLARSONHAND
0    17  438531560821  201712               1           40.0
1   239  438550152328  201719               2           28.8

So you want to change the header in the file, on the file system? — Chris, Feb 17 '17 at 15:24
There are certainly easier ways to do this than Pandas or even Python. — miradulo, Feb 17 '17 at 15:25
It seems you have to rewrite file - [info](http://stackoverflow.com/a/16306879/2901002) — jezrael, Feb 17 '17 at 15:30
This is best suited for commandline-like, shell script, instead of using python/pandas just for this. — Zero, Feb 17 '17 at 15:40

miradulo · Answer 1 · 2017-02-17T15:37:22.867

1

Just specify there being only a single row with nrows.

header_df = pd.read_csv('my_file.csv', index_col=0, nrows=1)

As for re-writing the file, I don't think you'll get around having to process the entire file to re-write.

edited Feb 17 '17 at 15:37

answered Feb 17 '17 at 15:19

miradulo

28,857
6
80
93

Python Pandas - Write New CSV Header Row without Reading/ReWriting Entire File

1 Answers1