Comparing arrays in shell

Question

I am writing a shell script as below, that gets a list of files provided by the user in a file and then ftp's to a server and then compares the list of files to what is on the server. The issue I am having is that when I am calling my diff function, the list that is being returned are the files that are unique to both arrays. I want only those that are in unique_array1 but not in unique_array2. In short, a list that shows what files within the list the user provided are not on the ftp server. Please note, that in the list of files provided by the user, each line is a file name, separated by a new line character.My script is as below:

    #!/bin/bash

    SERVER=ftp://localhost
    USER=anonymous
    PASS=password
    EXT=txt
    FILELISTTOCHECK="ftpFileList.txt"

    #create a list of files that is on the ftp server
    listOfFiles=$(curl $SERVER --user $USER:$PASS 2> /dev/null | awk '{ print $9 }' | grep -E "*.$EXT$")

    #read the list of files from the list provided##
    #Eg:
    # 1.txt
    # 2.txt
    # 3.txt
    #################################################
    listOfFilesToCheck=`cat $FILELISTTOCHECK`

    unique_array1=$(echo $listOfFiles | sort -u)
    unique_array2=$(echo $listOfFilesToCheck | sort -u)
    diff(){
     awk 'BEGIN{RS=ORS=" "}
      {NR==FNR?a[$0]++:a[$0]--}
      END{for(k in a)if(a[k])print k}' <(echo -n "${!1}") <(echo -n "${!2}")
    }

    #Call the diff function above
    Array3=($(diff unique_array1[@] unique_array2[@]))

    #get what files are in listOfFiles but not in listOfFilesToCheck
    echo ${Array3[@]}

score 0 · Answer 1 · edited May 23 '17 at 11:57

Based on this You may try comm command:

Usage: comm [OPTION]... FILE1 FILE2
Compare sorted files FILE1 and FILE2 line by line.

With no options, produce three-column output.  Column one contains
lines unique to FILE1, column two contains lines unique to FILE2,
and column three contains lines common to both files.

  -1              suppress column 1 (lines unique to FILE1)
  -2              suppress column 2 (lines unique to FILE2)
  -3              suppress column 3 (lines that appear in both files)

A test program:

#!/bin/bash

declare -a arr1
declare -a arr2

arr1[0]="this"
arr1[1]="is"
arr1[2]="a"
arr1[3]="test"

arr2[0]="test"
arr2[1]="is"

unique_array1=$(printf "%s\n" "${arr1[@]}" | sort -u)
unique_array2=$(printf "%s\n" "${arr2[@]}" | sort -u)

comm -23 <(printf "%s\n" "${unique_array1[@]}") <(printf "%s\n" "${unique_array2[@]}")

Output:

a
this

Comparing arrays in shell

1 Answers1