How to substitute bash variables into a yq command within a bash function

Question

Say I have an example YAML file, example.yaml:

services:
test:
  item0:
  item1:
  item2:
  item3:

And I have the follow bash function to recurse through the YAML levels:

function readYAML {
    local yaml="$1"
    declare -a arr
    readarray -t arr <<< "$(echo "$yaml" | yq -r '. |select(.) |  keys | .[]' )"
    echo "array -> ${arr[@]}"
    for item in "${arr[@]}"; do
        if [$item != ""]; then
            echo $("'.$item'")
            yq -r "$("'.$item'")" | readYAML;
        fi
    done
}

After running

readYAML "$(cat example.yaml)"

I get a yq error like '.test': command not found

How can I adjust the yq -r $("'.$item'") | readYAML; line so that I end up with an array arr that contains item0 ... item3?

I think that yq is interpreting $("'.$item'") as a command not as a filtering instruction. So I am doing the variable substitution incorrectly somehow.

N.B.

i. I know arr is overwritten (just stripped out array naming to simplify the question).

ii. I am using Go yq (Mike Farrah v4.31.2)

iii. There are similar questions. But none that I found dealing with substitute bash variables into a yq command within a bash function like this question:

Pass bash variable in yq

score 1 · Answer 1 · answered Mar 11 '23 at 11:00

1

Which implementation of yq are you using?

If you are using kislyuk/yq, you can make use of the @sh builtin from stedolan/jq (which is used under the hood) to properly escape the exported strings for shell, and declare your array at once:

$ unset arr; declare -a arr="($(yq -r '.test | keys_unsorted | @sh' example.yaml))"
$ echo "${arr[1]}"
item1

mikefarah/yq also implemented an @sh encoder in the quite recent version v4.31.1, so now you can do the same:

$ unset arr; declare -a arr="($(yq -r '.test | keys | .[] | @sh' example.yaml))"
$ echo "${arr[1]}"
item1

answered Mar 11 '23 at 11:00

pmf

24,478
2
22
31

Thanks this is an improvement to the `readarray` line I think. However, where I'm struggling is in recursively calling the function to get the next level down. In this case in passing `item0: item1:...` etc as an argument to `readYAML`. – Lee Mar 11 '23 at 11:10
Using mikefarah/yq at the moment btw (question updated) – Lee Mar 11 '23 at 11:11
1

@Lee My approach handles the whole array at once, so you wouldn't need a (recursive) function. If you want to do it iteratively, I would still opt for a loop passing in incremented indices for the `keys` array (`bash` is heavily imperative, after all). when using a bash function (even more so in the recursive case) you also have to somehow handle the "returning" of the array wrt scoping and subshells. – pmf Mar 11 '23 at 11:20
1

@Lee As a sidenote: If you wanted to "live" in a more declarative-styled shell, have a look at https://www.nushell.sh/. It implements arrays and can natively read YAML. My approach from above would translate to `let arr = (open example.yaml | get test | columns)`, then printing the array would be simply `echo $arr`, and single items just `echo $arr.1`. Also, recursive functions work just as you'd expect them to. – pmf Mar 11 '23 at 11:58

How to substitute bash variables into a yq command within a bash function

1 Answers1