Exchange columns in bash

Question

I have this file:

$ cat file
1515523  A45678BF141  A11269151
2234545  A45678BE145  A87979746
5432568  A45678B2123  A40629187
7234573  A45678B4154  A98879129
8889568  A45678B5123  A13409137
9234511  A45678B9176  A23589941
3904568  A45678B7123  A52329165
3234555  A45678B1169  A23589497
9643568  A45678B6123  A39969112
1234547  A45678B2132  A40579243

and this script:

cat file | awk '{FS = " "} {print $1" "$3" "$5}'| awk '{
    n = split($3, a, "");
    s = "";
    for (i = 1; i <= n; i += 2) s = s a[i+1] a[i];
    print $1, substr($2, length($2)-3, 4), s
}'| cut -d" " -f3,1  >  output

And when I open the output with vi, I have:

1515523  F141  11621915^M
2234545  E145  78797964^M
5432568  2123  04261978^M
7234573  4154  89781992^M
8889568  5123  31041973^M
9234511  9176  32859914^M
3904568  7123  25231956^M
3234555  1169  32854979^M
9643568  6123  93691921^M
1234547  2132  04752934^M

I don't know why I obtain ^M, because when I intend to run the awk snippet:

cat imei | awk '{FS=" "} {print $2","$1}'

the output is mistaken, i.e., it does not exchange the columns, as it does not print the second column. Any ideas on what may be happening?

There's no need to pipe `cat` into `awk`, `awk` can take a filename as an argument. Also the default field separator is one or more whitespaces, so there's no need to set `FS` and run the first `awk` command. — ThisSuitIsBlackNot, Aug 12 '13 at 16:37
The script you posted couldn't possibly have created that output from the input you provided. — Ansgar Wiechers, Aug 12 '13 at 17:09

score 3 · Answer 1 · edited May 23 '17 at 12:20

There are carriage returns (^M or Control-M) in the data file. It probably came from a Windows machine at some point.

When you print $2","$1 (which concatenates $2 with a string containing a comma and then $1 — it took me a couple of looks to see what it was really doing), the carriage return makes the second column overwrite the first.

Look at the data file with od -c or similar tools to see the carriage returns in it.

You can use dos2unix or tr or various other techniques to convert the file from DOS/Windows format to Unix format.

Also, given the data format shown, I'd expect not to use -F " " (or the FS = " ", which is equivalent), so that you have columns $1, $2, and $3, which is more obvious than working with columns 1, 3, 5 as shown. You could set OFS to double-blank if you wanted the output with two blanks between columns.

score 2 · Answer 2 · answered Aug 12 '13 at 17:33

$ dos2unix file
$ awk '{split($3,a,""); print $1, substr($2,8), a[3]a[2]a[5]a[4]a[7]a[6]a[9]a[8]}' file
1515523 F141 11621915
2234545 E145 78797964
5432568 2123 04261978
7234573 4154 89781992
8889568 5123 31041973
9234511 9176 32859914
3904568 7123 25231956
3234555 1169 32854979
9643568 6123 93691921
1234547 2132 04752934

score 0 · Answer 3 · answered Aug 13 '13 at 06:26

0

Since you are using awk you do not need a dos2unix.

simply insert

 gsub(/\r/,"");

as a first statement in your awk Script It cleans up each line read in. Subsequent matching or processing does not get any 'carriage return' characters.

answered Aug 13 '13 at 06:26

tue

497
2
8

score 0 · Answer 4 · answered Aug 13 '13 at 08:13

0

How about a perl 'one liner' (with a continuation line)

$ dos2unix file
$ perl -lane \
'$xxxx = substr($F[1],-4);
@c = split(//,$F[2]);
print "$F[0] $xxxx $c[2]$c[1]$c[4]$c[3]$c[6]$c[5]$c[8]$c[7]"' file

answered Aug 13 '13 at 08:13

AAAfarmclub

2,202
1
19
13

Exchange columns in bash

4 Answers4