How to print $1, $2 and then all remaining parameters together in AWK

Question

I'm making a script that generates aliases/abbreviations from a base file. The base file structure is something like this:

sctl   sudo         systemctl
pac    sudo         pacman

This works fine with the following code that reads the base file, removes comments and awks the abbreviation line on the abbreviation file:

    sed "s/\s*#.*$//;/^\s*$/d" $command_file | 
    awk -v c=$cmd -v o="$comp" '{ print c" "$1""o"\""$2" "$3"\"" }' >> $file

And the end result would be something like this:

abbr sctl "sudo systemctl"
abbr pac "sudo pacman"

But this code doesn't work when the line has many parts after the 3rd parameter:

svu    playerctl    -p spotify volume +0.05

How can i go about printing in that format? $1 $2 ($3..$N)

See https://stackoverflow.com/questions/25461806/how-to-print-a-range-of-columns-in-a-csv-in-awk — Wiktor Stribiżew, May 30 '19 at 12:06
I don't know how it would apply in this case, since I need to print either alias or abbrev at the start which is the c variable (for zsh or fish) and just after $1 I need to put either " " or "=" (again for fish or zsh) which is the o variable, and also quotes before $2 and after the last parameter. — Onizudo, May 30 '19 at 12:29
you could just get rid of the sed component and run the awk block on lines that don't start with comments / have at least so many fields — Rorschach, May 30 '19 at 12:53
Much easier simply to keep an `aliases` file and source it in your `.bashrc`. Suggestions, create separate blocks that test `if [ "$UID" -eq '0' ]; then alias pms='pacman -S --needed'; alias pmsu='pacman -Syu'; ... else alias pms='sudo pacman -S --needed'; alias pmsu='sudo pacman -Syu'; ... fi` — David C. Rankin, May 30 '19 at 13:17

Rorschach · Accepted Answer · 2019-05-30T15:35:59.967

2

You can erase the first two fields and trim the space from the remainder, eg.

{ 
  printf "%s %s ", $1, $2
  $1=$2=""; sub(/^\s*/, "", $0);
  printf "\"%s\"\n", $0
}

With output like,

svu playerctl "-p spotify volume +0.05"

Note: the \s regex requires gnu awk as pointed out by Ed Morton.

edited May 30 '19 at 15:35

answered May 30 '19 at 12:42

Rorschach

31,301
5
78
129

Ed Morton · Answer 2 · 2019-05-30T15:58:03.410

You never need sed when you're using awk. Given this input file:

$ cat file
sctl   sudo         systemctl   # here is a comment
        # and here is another
pac    sudo         pacman

svu    playerctl    -p spotify volume +0.05

here's what your sed+awk outputs:

$ sed "s/\s*#.*$//;/^\s*$/d" file |  awk -v c="abbr" -v o=" " '{ print c" "$1""o"\""$2" "$3"\"" }'
abbr sctl "sudo systemctl"
abbr pac "sudo pacman"
abbr svu "playerctl -p"

which you can get just from awk alone:

$ awk -v c="abbr" -v o=" " '{sub(/\s*#.*/,"")} NF{print c" "$1""o"\""$2" "$3"\""}' file
abbr sctl "sudo systemctl"
abbr pac "sudo pacman"
abbr svu "playerctl -p"

and which you can then adjust to do what you want:

$ awk -v c="abbr" -v o=" " '{sub(/\s*#.*/,"")} NF{x=$1; sub(/^\S+\s+/,""); $1=$1; print c, x o "\"" $0 "\""}' file
abbr sctl "sudo systemctl"
abbr pac "sudo pacman"
abbr svu "playerctl -p spotify volume +0.05"

or (I can't tell from your question what you want inside/outside of the double quotes):

$ awk -v c="abbr" -v o=" " '{sub(/\s*#.*/,"")} NF{x=$1" "$2; sub(/^(\S+\s+){2}/,""); $1=$1; print c, x o "\"" $0 "\""}' file
abbr sctl sudo "systemctl"
abbr pac sudo "pacman"
abbr svu playerctl "-p spotify volume +0.05"

The above uses GNU awk for \s and \S - with other awks use [[:space:]] and [^[:space:]] instead.

Since we're using GNU awk, though, we can do the job more concisely and efficiently with the 3rd arg to match():

$ awk -v c="abbr" -v o=" " '{sub(/\s*#.*/,""); $1=$1} match($0,/(\S+) (.*)/,a){print c, a[1] o "\"" a[2] "\""}' file
abbr sctl "sudo systemctl"
abbr pac "sudo pacman"
abbr svu "playerctl -p spotify volume +0.05"

$ awk -v c="abbr" -v o=" " '{sub(/\s*#.*/,""); $1=$1} match($0,/(\S+ \S+) (.*)/,a){print c, a[1] o "\"" a[2] "\""}' file
abbr sctl sudo "systemctl"
abbr pac sudo "pacman"
abbr svu playerctl "-p spotify volume +0.05"

I'm gonna try this later, thank you for giving me a new perspective, I thought you needed sed to do the comment part. Upvoted =D — Onizudo, May 30 '19 at 20:04

score 1 · Answer 3 · answered May 30 '19 at 12:30

Substitute the two extra fields out of existence.

$ echo "svu playerctl -p spotify volume +0.05" | gawk '
  { print $1; $1 = ""
    print $2; $2 = ""
    print substr($0,3) } '
svu
playerctl
-p spotify volume +0.05

The substr removes the output field separators from the remnants of the first two fields.

Dudi Boy · Answer 4 · 2019-05-30T15:32:51.237

1

Here is an awk (standard Linux - specific to gawk) script that does all the work in one sweep:

awk -v c="abbr" -v o=" " '/(^\s*$)|(^\s*#.*$)/ {next}  # discard empty lines and comments
{
    arg3=$0;                          # save current line to arg3 variable
    sub($1" "$2,"",arg3);             # remove $1  and $2 from arg3
    print(c" "$1""o"\""$2" "arg3"\"") # print formated output
}' input.txt

input.txt

ctl   sudo         systemctl
# comment line
pac    sudo         pacman
   # comment line
demo1  sudo         arg1 arg2 arg3

demo2  sudo         arg4 -arg5 -arg6 456
                         # comment line

Output:

abbr ctl "sudo          systemctl"
abbr pac "sudo          pacman"
abbr demo1 "sudo          arg1 arg2 arg3"
abbr demo2 "sudo          arg4 -arg5 -arg6 456"

edited May 30 '19 at 15:32

answered May 30 '19 at 12:47

Dudi Boy

4,551
1
15
30

I'm getting premature EOF when i use your approach – Onizudo May 30 '19 at 13:15
a symptom of unclosed `'` . test your input file `cat input.txt`, add a terminating new line. – Dudi Boy May 30 '19 at 13:17
Removed the ending } of the function when deleting the old stuff =P – Onizudo May 30 '19 at 13:24
thanks, please express your appreciation with an upvote. – Dudi Boy May 30 '19 at 14:37
That wouldn't remove comments from the end of a line that has data in it, e.g. something along the lines of `ctl sudo systemctl # comment at end of line`. Also avoid writing `sub($1...` as it'll fail when $1 contains RE metachars. use `sub(/\S...` instead (assuming removing $1 is what you want to do and you're using GNU awk for `\S`). – Ed Morton May 30 '19 at 16:03
1

Thanks, removing comments from end of line was not required. Thanks for the $1 comment, very correct. Following the current questions $1 and $2 are pure alpha. – Dudi Boy May 30 '19 at 16:08
Removing comments from end of line wasn't NOT required either :-). The OPs sed script handles comments stand-alone or at the end of lines so if you're handling one.... Yeah, I know the examples are just alphabetic but who knows what else is in the real files and avoiding using input strings in a regexp context (unless you have a very specific need) is a good habit to get into. – Ed Morton May 30 '19 at 16:50

P.... · Answer 5 · 2019-05-30T15:31:05.457

Here is GNU sed version:

sed -r 's/(^[^ ]+)\s+([^ ]+)\s+(.*)/abbr \1 "\2 \3"/g' base_file
abbr sctl "sudo systemctl"
abbr pac "sudo pacman"
abbr svu "playerctl -p spotify volume +0.05"

Explanation: Here back-referencing is used, each column is captured into respective groups and later used as \1, \2,\3. Note that -r flag is used in above sed command to enable extended regex. If this flag is not available in the target machine, then following command can be used:

sed  's/\(^[^ ]\+\)\s\+\([^ ]\+\)\s\+\(.*\)/abbr \1 "\2 \3"/g' base_file
abbr sctl "sudo systemctl"
abbr pac "sudo pacman"
abbr svu "playerctl -p spotify volume +0.05"

Only difference in the two commands in that ( ,),+ are escaped in the later.

How to print $1, $2 and then all remaining parameters together in AWK

5 Answers5