-3

I have a 1.5 GB .dat file that I need to import as a pandas dataframe, I'm running into memory issues (8GB RAM). How to I break the dat file into chunks to perform analysis?

anshumankarthikr
  • 11

1 Answers1

1
pd.read_csv('file.csv', chunksize = x)

returns an iterator giving x lines from file.csv at a time.

Ken Wei
  • 3,020
  • 1
  • 10
  • 30