Assigning space-separated computed values to array in bash is inconsistent with printing

Question

I had the following piece of az cli output in plain text:

echo $raw_containers_string
[
  {
    "name": "123"
  },
  {
    "name": "vbm-container"
  }
]

After some text refinement, I have a string returned containing this (zsh):

echo $raw_containers_string | grep name | cut -d ":" -f2 | tr '\n' " "
"123"  "vbm-container" %

(It also has a % symbol at the end, but that's expected)

I now need to create an array of these 2 strings (123 and vbm-container) to iterate through it.

1. declare -a arr=($(echo $raw_containers_string | grep name | cut -d ":" -f2 | tr '\n' " "))
   • returns "123" "vb -co t i r"
2. arr=($(echo $raw_containers_string | grep name | cut -d ":" -f2 | tr '\n' " "))
   • returns "123" "vb -co t i r"

These are indices of an array (if it matters):

➜  bash-az-list-blobs git:(master) ✗ echo $myvar[0]

➜  bash-az-list-blobs git:(master) ✗ echo $myvar[1]
 "123"  "vb
➜  bash-az-list-blobs git:(master) ✗ echo $myvar[2]
-co
➜  bash-az-list-blobs git:(master) ✗ echo $myvar[3]
t
➜  bash-az-list-blobs git:(master) ✗ echo $myvar[4]
i
➜  bash-az-list-blobs git:(master) ✗ echo $myvar[5]

➜  bash-az-list-blobs git:(master) ✗ echo $myvar[6]
r"

Questions

1. I want to understand why the behaviour between printing it to the terminal and assigning it to a variable is different.
2. I would also like to know how I assign my refined to an array in zsh, so that echo arr returns a 2-elements iterable array.

UPD (08-16)

Using jq does filter JSON successfully

echo "$raw_containers_string" | jq -r '.[].name'
123
vbm-container

The problem is to pack these values into an array (this was exactly the purpose of this thread):

declare -a arr=$(echo "$raw_containers_string" | jq -r '.[].name')
echo $arr
123 vbm-container
echo $arr[0]
123 vbm-container[0]
echo $arr[1]
123 vbm-container[1]

Why accessing the first element of my array just add index "[0]" to an output?

echo "$raw_containers_string" | jq -r '.[].name'
→ 123
vbm-container

declare -a containers_array=($(echo "$raw_containers_string" | jq -r '.[].name|@sh' | tr '\n' " "))
→ "123"

eval "containers_array=($(echo "$raw_containers_string" | jq -r '.[].name|@sh'))"
→ 123

eval "containers_array=($(echo "$raw_containers_string" | jq '.[].name|@sh'))"
→ '123'

echo $IFS
→ 

Answer 1

You're counting on word-splitting of unquoted variable/command substitutions to separate the elements of the array, and have apparently also changed IFS. These are both bad ideas, and together they're a recipe for chaos. You're also using zsh for interactive testing, and it does word splitting very differently from bash, so you'll get different weird results from it.

First, track down what's changing IFS and fix it. You appear to have it set to something really weird, containing at least "m", "n", "a", and "e", so it's probably set by mistake.

If it actually does need to be changed, you need to keep it from staying changed. You can either make the change local to a specific command by making it a prefix of the command (e.g. IFS=',' read field1 field2) or by resetting it after using the changed value (e.g. saveIFS="$IFS"; IFS=","; doSomethingWithIFS; IFS="$saveIFS"), or just find a different way to solve whatever you changed IFS` for in the first place.

Second, get in the habit of double-quoting variable and command substitutions. There are some specific situations where you shouldn't double-quote them, and some situations where it's optional, but it's safest to just assume it should be double-quoted in most cases. shellcheck.net is good at pointing out quoting problems (and many other common mistakes).

Instead of using word-splitting on an unquoted substitution to populate the array, use readarray -t to read each line of input into a separate array element (or read -ra to read words into elements -- but fix IFS first, or it'll also split weirdly). Note that you can't put these in a pipeline, or they execute in a subshell and the value gets lost; instead, use process substitution: readarray -t < <(somecommand)

Third, that grep | cut thing is going to output something with extra spaces and quotes around the actual values. Anyway, when parsing JSON, it's better to use a tool like jq that actually speaks the language.

So, with all that fixed, try this:

readarray -t arr < <(echo "$raw_containers_string" | jq -r '.[].name')

Léa Gris pointed out that you can simplify this even further by passing the string to jq with --argjson instead of echo and a pipe:

readarray -t arr < <(jq -rn --argjson j "$raw_containers_string" '$j[].name')

If you have an older version of bash that doesn't support readarray (i.e. you're on macOS), you can fake it with read and some delimiter switching:

IFS=$'\n' read -d '' -a arr < <(jq -rn --argjson j "$raw_containers_string" '$j[].name')

Here, the -d '' part makes it treat the entire thing as one big line, and IFS=$'\n' tells it to break that "line" into fields on newline delimiters. Since the IFS assignment is a prefix to the read command, it only applies to that one command and won't cause trouble later. One warning, though: technically, the read command will exit with an error status (because it never saw an end-of-line delimiter), so if you use set -e or shopt -s errexit it'll make the script exit (add || true at the end to prevent this).

(And if you need to check the contents of an array -- or even a plain variable -- declare -p arr is more reliable than echo.)

Answer 2

Using jq:

eval "names=($(echo "$raw_containers_string" |
               jq -r '.[].name|@sh'))"

Now you have a shell array names, where each name is an element. Piping to @sh means each value is quoted as a separate shell word, safe to eval.