issue for condition on unique raws in bash

Question

I want to print rows of a table in a file, the issue is when I use a readline the reprint me the result several times, here is my input file

aa      ,DEC    ,file1.txt
aa      ,CHAR   ,file1.txt    
cc      ,CHAR   ,file1.txt  
dd      ,DEC    ,file2.txt
bb      ,DEC    ,file3.txt
bb      ,CHAR   ,file3.txt 
cc      ,DEC    ,file1.txt

Here is the result I want to have:

printed in file1.txt

aa#DEC,CHAR
cc#CHAR,DEC

printed in file2.txt

dd#DEC

printed in file3.txt

bb#DEC,CHAR

here is it my attempt :

(cat input.txt|while read line
do
table=`echo $line|cut -d"," -f1
variable=`echo $line|cut -d"," -f2
file=`echo $line|cut -d"," -f3

echo ${table}#${variable}, 

done ) > ${file}

You need to redirect echo to the relevant file within the loop. At the moment, all of the output is going to one file. — Raman Sailopal, Dec 10 '20 at 14:43
Please copy/paste your script into http://shellcheck.net (as instructed in the bash tag info, https://stackoverflow.com/tags/bash/info), fix the issues it tells you about, read [why-is-using-a-shell-loop-to-process-text-considered-bad-practice](https://unix.stackexchange.com/questions/169716/why-is-using-a-shell-loop-to-process-text-considered-bad-practice) and then re-post if you still have a question. — Ed Morton, Dec 10 '20 at 15:53

score 2 · Answer 1 · answered Dec 10 '20 at 15:56

This can be done in a single pass gnu awk like this:

awk -F ' *, *' '{
   map[$3][$1] = (map[$3][$1] == "" ? "" : map[$3][$1] ",") $2
}
END {
   for (f in map)
      for (d in map[f])
         print d "#" map[f][d] > f
}' file

This will populate this data:

=== file1.txt ===

aa#DEC,CHAR
cc#CHAR,DEC

=== file2.txt ===

dd#DEC

=== file3.txt ===

bb#DEC,CHAR

RavinderSingh13 · Accepted Answer · 2020-12-10T15:28:33.930

With your shown samples, could you please try following, written and tested in shown samples in GNU awk.

awk '
{
  sub(/^,/,"",$3)
}
FNR==NR{
  sub(/^,/,"",$2)
  arr[$1,$3]=(arr[$1,$3]?arr[$1,$3]",":"")$2
  next
}
(($1,$3) in arr){
  close(outputFile)
  outputFile=$3
  print $1"#"arr[$1,$3] >> (outputFile)
  delete arr[$1,$3]
}
'  Input_file  Input_file

Explanation: Adding detailed explanation for above.

awk '                         ##Starting awk program from here.
{
  sub(/^,/,"",$3)             ##Substituting starting comma in 3rd field with NULL.
}
FNR==NR{                      ##Checking condition FNR==NR will be true when first time Input_file is being read.
  sub(/^,/,"",$2)             ##Substituting starting comma with NULL in 2nd field.
  arr[$1,$3]=(arr[$1,$3]?arr[$1,$3]",":"")$2 
##Creating arr with index of 1st and 3rd fields, which has 2nd field as value.
  next                        ##next will skip all further statements from here.
}
(($1,$3) in arr){             ##Checking condition if 1st and 3rd fields are in arr then do following.
  close(outputFile)           ##Closing output file, to avoid "too many opened files" error.
  outputFile=$3               ##Setting outputFile with value of 3rd field.
  print $1"#"arr[$1,$3] >> (outputFile)
##printing 1st field # arr value and output it to outputFile here.
  delete arr[$1,$3]           ##Deleting array element with index of 1st and 3rd field here.
}
' Input_file Input_file       ##Mentioning Input_file 2 times here.

it's really working but can you give more explanation about Line 7 in your code ? what means the "?" and ,",":"")$2 in tihs line and also line 14 please,thank you — BADS, Dec 10 '20 at 21:50
@BADS, your welcome, happy that it helped you. it uses ternary operators. its syntax is like: `condition_check?value(when condition is TRUE):value(when condition is false)` format. `arr[$1,$3]` means to check if its NOT NULL then append current line's 2nd field value to its existing value(as per requirement) OR add it newly to array as array will be empty. I hope its clear now, cheers. — RavinderSingh13, Dec 11 '20 at 03:38

score 0 · Answer 3 · answered Dec 10 '20 at 15:21

You have several errors in your code. You can use the built-in read to split on a comma, and the parentheses are completely unnecessary.

while IFS=, read -r table variable file
do
    echo "${table}#${variable}," >>"$file"
done< input.txt

Using $file in a redirect after done is an error; the shell wants to open the file handle to redirect to before file is defined. But as per your requirements, each line should go to a different `file.

Notice also quoting fixes and the omission of the useless cat.

Wrapping fields with the same value onto the same line would be comfortably easy with an Awk postprocessor, but then you might as well do all of this in Awk, as in the other answer you already received.

issue for condition on unique raws in bash

3 Answers3