1

Combine sed commands into one command

I am currently doing these two commands

removes the first character of each line

sed -i 's/\(.\{1\}\)//'

removes extra spaces in each line

sed -i 's/    / /g'

There are 3.4 BILLION lines in the 237GB file it is parsing, and i dont want it to need to run through twice.

BulletB
  • 49
  • 7
  • Command will look like sed -i 's/\(.\{1\}\)//;'s/ / /g' Found out you just need to use a semicolon. Sorry for clogging up boards. – BulletB Jun 30 '14 at 17:36
  • 1
    How is the first command different from `sed -i 's/.//'`? – Beta Jun 30 '14 at 17:36
  • Honestly don't know, found the command on a previous stackoverflow question and it worked so i didnt question it. – BulletB Jun 30 '14 at 17:38
  • 2
    The `{1}` quantifier is completely pointless. You probably got it from some PHP guru... – tripleee Jun 30 '14 at 18:20

3 Answers3

2

The below sed command would combine the both. Use ; as separator to combine two sed operations.

sed -i 's/\(.\{1\}\)//;s/    / /g' file
Avinash Raj
  • 172,303
  • 28
  • 230
  • 274
2

Another way:

sed  -i -e 's/\(.\{1\}\)//' -e 's/    / /g'  file
Tiago Lopo
  • 7,619
  • 1
  • 30
  • 51
1

You can try an awk

awk '{sub(/./,"");$1=$1}1' file

sub(/./,"") removes first character
$1=$1 removes all double space.

Jotne
  • 40,548
  • 12
  • 51
  • 55
  • is one way inherently faster than another? I am going for speed because of the size of the file. – BulletB Jun 30 '14 at 18:16
  • Also can awk edit files in place? Doesn't help me too much if it cant because the I/O time to copy the file is to high. – BulletB Jun 30 '14 at 18:33
  • `awk` does not do in place edit. You can do `awk 'code' file > tmp & mv tmp file`. For speed, you need to test. – Jotne Jun 30 '14 at 18:36