4

I want to extract and combine a certain column from a bunch of text files into a single file as shown.

File1_example.txt

A   123   1
B   234   2
C   345   3
D   456   4

File2_example.txt

A   123   5
B   234   6
C   345   7
D   456   8

File3_example.txt

A   123   9
B   234   10
C   345   11
D   456   12

...

..

.

File100_example.txt

A   123   55
B   234   66
C   345   77
D   456   88

How can I loop through my files of interest and paste these columns together so that the final result is like below without having to type out 1000 unique file names?

1   5   9   ...  55
2   6   10  ...  66
3   7   11  ...  77
4   8   12  ...  88
Jay
  • 741
  • 10
  • 26

3 Answers3

2

Try this:

paste File[0-9]*_example.txt | awk '{i=3;while($i){printf("%s ",$i);i+=3}printf("\n")}'

Example:
File1_example.txt: 

A   123   1
B   234   2
C   345   3
D   456   4

File2_example.txt:

A   123   5
B   234   6
C   345   7
D   456   8

Run command as: 

$ paste File[0-9]*_example.txt | awk '{i=3;while($i){printf("%s ",$i);i+=3}printf("\n")}'

Output:

1 5 
2 6 
3 7 
4 8
Ren
  • 2,852
  • 2
  • 23
  • 45
1

I tested below code with first 3 files

cat File*_example.txt  | awk '{a[$1$2]= a[$1$2] $3 " "} END{for(x in a){print a[x]}}' | sort

1 5 9
2 6 10
3 7 11
4 8 12

1) use an awk array, a[$1$2]= a[$1$2] $3 " " index is column1 and column2, array value appends all column 3.

2) END{for(x in a){print a[x]}} travesrsed array a and prints all values.

3)use sort to sort the output.

Haifeng Zhang
  • 30,077
  • 19
  • 81
  • 125
1

when cating you need to ensure the file order is preserved, one way is to explicitly specify the files

cat File{1..100}_example.txt | awk '{print $NF}' | pr 4ts' '

extract last column by awk and align using pr

karakfa
  • 66,216
  • 7
  • 41
  • 56