evaluate a command in a while loop in bash

Question

I am trying to write a code that runs a script on a list of files determined based on users input. for some reason the following code doesn't work? is there any way to do evalute the query_cmd and iterate over the files it outputs.

if [[ $# -gt 0 && "$1" == "--diff" ]]; then
  query_cmd="git diff --name-only '*.cc' '*.h'"
else
  query_cmd='find . \( -name "*.cc" -o -name "*.h" \)'
fi

while IFS='' read -r line; do
  absolute_filepath=$(realpath "$line")
  if [[ $absolute_filepath =~ $ignore_list ]]; then
     continue
  fi
  cpp_filepaths+=("$absolute_filepath")
done < <($query_cmd)

Storing complex commands in variables doesn't work. Either use an array instead, or just put the entire `if` block inside the `< <(...)` expression. See ["Why does shell ignore quoting characters in arguments passed to it through variables?"](https://stackoverflow.com/questions/12136948/why-does-shell-ignore-quoting-characters-in-arguments-passed-to-it-through-varia) and [BashFAQ #50: I'm trying to put a command in a variable, but the complex cases always fail!](http://mywiki.wooledge.org/BashFAQ/050) — Gordon Davisson, Mar 22 '21 at 23:43
@MrR, ...that's badly broken, for the reasons given in the FAQ Gordon linked just above your comment. For example, it'll read `'*.cc'` to mean that filenames need to begin and end with a `'`. — Charles Duffy, Mar 22 '21 at 23:57
@MrR, no, it won't work in the OP's case for the reason I described. Having the literal text `'*.cc'` in the `$query_cmd` string means the command needs quotes to be honored when it's parsed. When they're _not_ parsed, they're instead treated as literal characters. — Charles Duffy, Mar 23 '21 at 00:00
git _expects_ a command line of the form `["git", "diff", "--name-only", "*.cc", "*.h"]` (JSON-escaped), which is what a shell will do when it parses and executes `git diff --name-only '*.cc' '*.h'` as code. Running `$query_cmd` will instead run `["git", "diff", "--name-only", "'*.cc'", "'*.h'"]`, adding _literal_ single quotes that in a normally-parsed command would be removed by the shell. — Charles Duffy, Mar 23 '21 at 00:04

score 2 · Accepted Answer · answered Mar 22 '21 at 23:44

2

Generally if you have code you want to run later, you would put it in a function and not in a string:

if [[ $# -gt 0 && "$1" == "--diff" ]]; then
  query_cmd() {
    git diff --name-only '*.cc' '*.h'
  }
else
  query_cmd() {
    find . \( -name "*.cc" -o -name "*.h" \)
  }
fi

while IFS='' read -r line; do
  ...
done < <(query_cmd)

But if you enjoy the additional escaping challenges, you can use strings and evaluate them with eval:

if [[ $# -gt 0 && "$1" == "--diff" ]]; then
  query_cmd="git diff --name-only '*.cc' '*.h'"
else
  query_cmd='find . \( -name "*.cc" -o -name "*.h" \)'
fi

while IFS='' read -r line; do
  ...
done < <(eval "$query_cmd")

answered Mar 22 '21 at 23:44

that other guy

116,971
11
170
194

I'd consider showcasing use of arrays to store command lines long before demonstrating `eval`. (Be happy to edit towards that end myself, if it'd be welcome). – Charles Duffy Mar 22 '21 at 23:59
I don't know... I included the (arguably) best solution for the problem OP should have asked, and for the literal question they did ask. Arrays seem to be the worst of both worlds in this specific case – that other guy Mar 23 '21 at 00:13
And can do the function method with the if in the function too rather than having 2 variants .. – Mr R Mar 23 '21 at 00:14
@thatotherguy, re: arrays-as-worst-of-both-worlds, how so? `query_cmd=( git diff --name-only '*.cc' '*.h' )` and then `< <("${query_cmd[@]}")`... what's unclean? You don't have the extra escaping challenges, and you can modify your commands use variables in their arguments without introducing security bugs. – Charles Duffy Mar 23 '21 at 00:15
...similarly, in the other side of the `if`, `query_cmd=( find . '(' -name '*.cc' -o -name '*.h' ')' )` -- same escaping that one would use running that `find` command on a command line. – Charles Duffy Mar 23 '21 at 00:18
This requires changing the definition of the command just like the function approach, but unlike a function it puts additional restrictions on what can and can't be part of the command – that other guy Mar 23 '21 at 00:18
The advantage of the array - in memory .. the disadvantage of the array - in memory / less pipe-like (so slightly more obscure?) .. Don't get me wrong I like arrays but .... – Mr R Mar 23 '21 at 00:22
Arrays are way better than `eval` in this case, but people keep posting memes about how SO users always say "no, do this instead" so I try to include both a helpful answer and a verbatim answer. – that other guy Mar 23 '21 at 00:36

evaluate a command in a while loop in bash

1 Answers1