awk: log processing based on multiple patterns

Question

I am working with the log filles consisted of some measurements taken from different samples (identified as float numbers 1.1, 1.2 ... 1.14) that are arranged in the following format:

Finding intramodel H-bonds
Constraints relaxed by 0.5 angstroms and 20 degrees
Models used:
    1.1 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.2 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.3 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.4 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.5 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.6 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.7 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.8 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.9 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.10 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.11 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.12 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.13 SarsCov2_structure19R_nsp5holo_rep1.pdb
    1.14 SarsCov2_structure19R_nsp5holo_rep1.pdb

16 H-bonds
H-bonds (donor, acceptor, hydrogen, D..A dist, D-H..A dist):
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.1/? HIS 163 NE2   SarsCov2_structure19R_nsp5holo_rep1.pdb #1.1/A UNL 888 S   no hydrogen                                                   3.850  N/A
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.1/? GLU 166 N     SarsCov2_structure19R_nsp5holo_rep1.pdb #1.1/A UNL 888 O   SarsCov2_structure19R_nsp5holo_rep1.pdb #1.1/? GLU 166 H      2.909  2.070
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.1/A UNL 888 N     SarsCov2_structure19R_nsp5holo_rep1.pdb #1.1/? CYS 44 O    SarsCov2_structure19R_nsp5holo_rep1.pdb #1.1/A UNL 888 H      2.798  1.892
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.2/? GLN 189 NE2   SarsCov2_structure19R_nsp5holo_rep1.pdb #1.2/A UNL 888 S   SarsCov2_structure19R_nsp5holo_rep1.pdb #1.2/? GLN 189 1HE2   3.896  2.916
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.3/? GLU 166 N     SarsCov2_structure19R_nsp5holo_rep1.pdb #1.3/A UNL 888 O   SarsCov2_structure19R_nsp5holo_rep1.pdb #1.3/? GLU 166 H      2.673  1.892
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.3/A UNL 888 N     SarsCov2_structure19R_nsp5holo_rep1.pdb #1.3/? CYS 44 O    SarsCov2_structure19R_nsp5holo_rep1.pdb #1.3/A UNL 888 H      3.071  2.338
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.4/? HIS 163 NE2   SarsCov2_structure19R_nsp5holo_rep1.pdb #1.4/A UNL 888 S   no hydrogen                                                   3.927  N/A
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.4/A UNL 888 N     SarsCov2_structure19R_nsp5holo_rep1.pdb #1.4/? THR 190 O   SarsCov2_structure19R_nsp5holo_rep1.pdb #1.4/A UNL 888 H      3.029  2.173
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.8/? GLN 189 NE2   SarsCov2_structure19R_nsp5holo_rep1.pdb #1.8/A UNL 888 S   SarsCov2_structure19R_nsp5holo_rep1.pdb #1.8/? GLN 189 2HE2   3.631  2.751
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.9/? CYS 145 N     SarsCov2_structure19R_nsp5holo_rep1.pdb #1.9/A UNL 888 O   SarsCov2_structure19R_nsp5holo_rep1.pdb #1.9/? CYS 145 H      2.966  2.210
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.9/A UNL 888 N     SarsCov2_structure19R_nsp5holo_rep1.pdb #1.9/? ARG 188 O   SarsCov2_structure19R_nsp5holo_rep1.pdb #1.9/A UNL 888 H      3.067  2.307
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.10/? GLN 189 NE2  SarsCov2_structure19R_nsp5holo_rep1.pdb #1.10/A UNL 888 S  SarsCov2_structure19R_nsp5holo_rep1.pdb #1.10/? GLN 189 2HE2  3.693  2.786
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.11/A UNL 888 N    SarsCov2_structure19R_nsp5holo_rep1.pdb #1.11/? THR 190 O  SarsCov2_structure19R_nsp5holo_rep1.pdb #1.11/A UNL 888 H     3.159  2.268
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.12/? GLU 166 N    SarsCov2_structure19R_nsp5holo_rep1.pdb #1.12/A UNL 888 O  SarsCov2_structure19R_nsp5holo_rep1.pdb #1.12/? GLU 166 H     2.648  1.817
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.13/A UNL 888 N    SarsCov2_structure19R_nsp5holo_rep1.pdb #1.13/? THR 190 O  SarsCov2_structure19R_nsp5holo_rep1.pdb #1.13/A UNL 888 H     3.176  2.395
SarsCov2_structure19R_nsp5holo_rep1.pdb #1.14/A UNL 888 N    SarsCov2_structure19R_nsp5holo_rep1.pdb #1.14/? PHE 140 O  SarsCov2_structure19R_nsp5holo_rep1.pdb #1.14/A UNL 888 H     2.833  1.955

I need to print the number assosiated with the sample (1-14) that should be correspond to the first occurence of two patterns: the "GLU 166 N" as well as "CYS 44 O" and no other patterns within the same sample. I need to print the number present on the same line just before the pattern as #1.number/?, associated with this pattern. So in the example the detected number should be 3 (since the associating number is #1.3/?) where the both patterns (and no others!) could be found. Finally if the both patterns could not be found I would like to print the number corresponded to the sample with the first pattern "GLU 166 N" (like in my example)

Presently my AWK solution is focused on one pattern-based search: looking the first occurence of the "GLU 166 N" ( in the case if the pattern can not be found the script prints 1 ). Basically, it looks for the "pattern" anywhere on the line, and then prints the second part of the number (after the dot) from the 2nd field":

awk -vn=1 '/GLU 166 N/ {gsub(/.*\.|\/\?/,"",$2); n=$2; exit} END {print n}' input.log

Answer 1

Based our meaningful discussion, would you please try:

awk -F# '                               # split line on '#' into fields
{
    for (i = 1; i <= NF; i++) {         # loop over the fields
        if (match($i, /^1\.[0-9]+\/\? GLU 166 N/)) {
            sub(/^1\./, "", $i); sub(/\/.*/, "", $i)
                                        # extract the number after "1." in $i
            if (first == "") first = $i # keep the first found value as a fallback
            if ($i in b) {              # if the number exists also in b
                queue[++qn] = $i        # then push it in the queue
            }
            a[$i]
            next
        } else if (match($i, /^1\.[0-9]+\/\? CYS 44 O/)) {
            sub(/^1\./, "", $i); sub(/\/.*/, "", $i)
            if ($i in a) {
                queue[++qn] = $i
            }
            b[$i]
            next
        } else if (match($i, /^1\.[0-9]+\/\? [A-Z]{3} [0-9]+ [A-Z][A-Z0-9]*/)) {
                                        # analyse other patterns
            sub(/^1\./, "", $i); sub(/\/.*/, "", $i)
            exclude[$i]
        }
    }
}
END {
    for (i = 1; i <= qn; i++) {         # examine the queue in appearance order
        j = queue[i]                    # j is the matched number
        if (! (j in exclude)) {         # if not found in other patterns
            print j                     # then it is the answer
            exit
        }
    }
    if (first == "") print "1"          # the default value
    else print first                    # the fallback
}' input.log

• It searches for the names: GLU 166 N, CYS 44 O and other substances as well as the associated numbers embedded in the leading form #1.<num>/?.
• If both GLU 166 N and CYS 44 O have the same number, the number is pushed in queue in appearance order.
• We need to eliminate numbers which also appear with other substances (except for the case it appears in the same line after either of the two). The array exclude memorizes the numbers associated with these substances.
• In the END block we examine the numbers in queue in order. The first number in the queue which is not included in exclude will be used as the answer.
• If GLU 166 N and CYS 44 O do not have the same number, the first found number with GLU 166 N is used as a fallback.
• As a last resort, 1 will be used in case no pattern is found.

[EDIT]
Here is a one liner to use bash variables as the search patterns and assign a bash variable var to the output:

search_pattern1='GLU 166 N'
search_pattern2='CYS 44 O'

var=$(awk -F# -v pat1="$search_pattern1" -v pat2="$search_pattern2" '{for (i = 1; i <= NF; i++) {if (match($i, "^1\\.[0-9]+\\/\\? "pat1)) {sub(/^1\./, "", $i); sub(/\/.*/, "", $i); if (first == "") first = $i; if ($i in b) {queue[++qn] = $i} a[$i]; next} else if (match($i, "^1\\.[0-9]+\\/\\? "pat2)) {sub(/^1\./, "", $i); sub(/\/.*/, "", $i); if ($i in a) {queue[++qn] = $i} b[$i]; next} else if (match($i, /^1\.[0-9]+\/\? [A-Z]{3} [0-9]+ [A-Z][A-Z0-9]*/)) { sub(/^1\./, "", $i); sub(/\/.*/, "", $i); exclude[$i]}}} END {for (i = 1; i <= qn; i++) {j = queue[i]; if (! (j in exclude)) {print j; exit}} if (first == "") print "1"; else print first}' input.log)

[Explanations]
When using the variable as the regex pattern in awk, we need to take care of the quoting. In many cases we will use the statement as:

if (match($0, /regex/)) ...

where the slashes are used as quotes to enclose the regex pattern. Bare words within the quotes are treated as literal strings, not the variable name. That is why we cannot put the variable name within the quotes. For instance, if we say:

if (match($i, /^1\.[0-9]+\/\? pat1/)) ...

the word pat1 is no longer the variable name. It's just a literal string.

How can we solve it? We need to put the quoted string and the variable side by side so the awk concatenate them into a single pattern:

if (match($i, "^1\\.[0-9]+\\/\\? "pat1)) ...

• In this case we cannot use the slash quotes. Instead we need to use double quotes.
• We need to add another backslash to escape the backslash within the double quotes.

BTW your post "How can I use bash variables in my Awk script?" has unfortunately been closed due to the duplicate. However the essential problem is not there. I'm afraid the reviewers did not understand what you want to do.