3

Possible Duplicates:
Text Editor for very big file - Windows
What editor/viewer to use to inspect large text based files?

I have a CSV file which has about 4 million rows and is about 500 MB in size. Can you recommend any editor that can open the file without making the system crawl? I tried EmEditor but it is complaining that there are too many characters in a single line.

Community
  • 1
Giorgi
  • 651

7 Answers7

7

Vim should work pretty well if you change a couple settings to turn off swap space and undo functionality: http://vim.wikia.com/wiki/VimTip611

Herbert Sitz
  • 336
4

You could give CSVed a try. It claims to load parts of the file on demand, so it might work for you.

afrazier
  • 23,505
3

I've used JujuEdit (free) to work with 2GB CSVs in the past. Handles 'em pretty well. What are you doing with the contents? Another option might be to make an MS Access linked table pointed to the file if you need to do some querying/sorting/filtering.

Chris_K
  • 8,851
3

Define "edit". How do you want to edit the file? Are you just viewing it, or do you want to make changes? Are they changes that you'd be making programmatically? If so, you may want to look at using awk, sed or Perl to do it.

As to "it is complaining that there are too many characters in a single line," it's probably that it's got line endings it's not able to handle. It would help if you'd tell us what platform you're on.

Andy Lester
  • 1,302
  • 7
  • 15
3

I tested all the editors suggested there and the only one that managed to open it (in a reasonable time) was UltraEdit.


Update

It turned out that the file did not have any line break in it (it used some other character for it) so that's why it was difficult to open it. I wrote a program which replaced all these chars with line break and EmEditor opened it in seconds.

Giorgi
  • 651
2

I believe that UltraEdit probably could. I've used it on superhuge Gig+ files and it didn't crash out, but really, locating whatever data you are looking for is likely to be a pain. Can you do whatever you need to do programmetrically? Since you said CSV, assume it is text data, so maybe Perl might be a good choice due to its string handling and decent RegEx.

Blackbeagle
  • 6,702
1

Have you tried Excel? It handled CSV files.

I'm not sure of Excel 2007's file limitation but Excel 2010 64 bit now supports spreadsheets up to 4GB in size.

Mike Fitzpatrick
  • 17,427