-1

Suppose that I have a large text file with 100M lines; What is the most effective way to delete first N lines of it, such as 50M lines.

As I tried, only opening this file with vim will take several minutes.

Are there any more effective ways to accomplish this?

Dylan Su
  • 5,975
  • 1
  • 16
  • 25
  • If you know which lines you want to preserve, why not just copy those out of the file into a new file, then delete the original? – Govind Parmar May 14 '19 at 02:28
  • @GovindParmar This way is fine. I wonder how to do this without opening the large file with vim. – Dylan Su May 14 '19 at 02:30

1 Answers1

5
tail -n +50000000 file > tmp && mv tmp file

If you don't have the storage to almost duplicate the input file then to edit it truly in place (i.e. not using a temp file like all of the command line "inplace" editing tools like sed, perl, etc. use, nor a buffer the size of your input file like ed uses) you could do:

bytes=$(head -50000000 file |wc -c)
dd if=file bs="$bytes" skip=1 conv=notrunc of=file
truncate -s "-$bytes" file

See https://stackoverflow.com/a/17331179/1745001 for more info on that last script.

Ed Morton
  • 188,023
  • 17
  • 78
  • 185