Loop through json variable

Question

I have a json variable which looks like this:

{
  "status": "closed",
  "host": "host1-availabe_zone_A"
}
{
  "status": "closed",
  "host": "host2-availabe_zone_B"
}
{
  "status": "closed",
  "host": "host3-availabe_zone_A"
}

I am trying to convert it to a hash-like data structure (associate array in zsh/bash) to get number of hosts in each availabe zone. The preferred result will be this based on the data above:

availabe_zone_A -> 2
availabe_zone_B -> 1

I got some problem when trying to iterate the json variable.

declare -A az_to_number_of_open_hosts
for host in $(jq . <<<  $hosts_list)
do
  az=$(rev <<< $(echo $host | jq .host) | cut -d. -f3 | rev)
  ((az_to_number_of_open_hosts[$az]++))
done

The line to extract available zone is because the host format is always blabla123213.az.domain.com. The code does not work since it seems the variable host refers to each line of json string rather than a json entry. Is there a recommended way to iterate each json entry?

That's not valid JSON. If you have multiple objects they have to be contained in an array so you can loop through it. — Barmar, Aug 28 '19 at 15:53
What are you expecting `jq .` to do? It just returns the input as its output. — Barmar, Aug 28 '19 at 15:56
Why do you declare `az_to_number_of_open_hosts` and not use it? — Barmar, Aug 28 '19 at 15:57
@Barmar Isn't it a valid json? since jq works pretty well with the entries above: `jq .host /tmp/test.json "host1-availabe_zone_A" "host2-availabe_zone_B" "host3-availabe_zone_A" ` — dashenswen, Aug 28 '19 at 16:00
seems the format does not work in comment :(. I use `echo` just want to show it only prints each line of string. And yes, I will do some extraction to extract the available zone from the `.host` and use it as a key of the associate array — dashenswen, Aug 28 '19 at 16:04
@Barmar update the code as your comment. Hope it makes sense to u — dashenswen, Aug 28 '19 at 16:07
[https://stackoverflow.com/questions/26717277/converting-a-json-object-into-a-bash-associative-array][1] — Bizzu, Aug 28 '19 at 16:19
`jq` can iterate over a *stream* of separate JSON values, which is what you have. — chepner, Aug 28 '19 at 16:49

score 1 · Answer 1 · answered Aug 28 '19 at 17:32

You're just counting values in your json. Split out the part you want to count

$ jq -nr 'inputs.host/"-"|last' input.json
availabe_zone_A
availabe_zone_B
availabe_zone_A

then you could either group and count:

$ jq -nr '[inputs.host/"-"|last]|group_by(.)[]|"\(.[0]) -> \(length)"' input.json
availabe_zone_A -> 2
availabe_zone_B -> 1

https://jqplay.org/s/g_8HgqiQ1o

or iterate the values tallying as you go:

$ jq -nr '
reduce (inputs.host/"-"|last) as $k ({}; .[$k] += 1)|to_entries[]|"\(.key) -> \(.value)"
' input.json
availabe_zone_A -> 2
availabe_zone_B -> 1

https://jqplay.org/s/S2oeLfMRNR

score 0 · Answer 2 · answered Aug 28 '19 at 16:05

0

Use .host to get the host properties.

$ hosts_list='  {
    "status": "closed",
    "host": "host1-availabe_zone_A"
  }
  {
    "status": "closed",
    "host": "host2-availabe_zone_B"
  }
  {
    "status": "closed",
    "host": "host3-availabe_zone_A"
  }'
$ for host in $(jq -r '.host' <<<  "$hosts_list")
do
 echo "$host"
done
host1-availabe_zone_A
host2-availabe_zone_B
host3-availabe_zone_A

Use the -r option to make it output raw strings, rather than JSON strings surrounded by double quotes.

And don't forget to quote variables unless you deliberately need word-splitting and wildcard matching to be done on the value.

answered Aug 28 '19 at 16:05

Barmar

741,623
53
500
612

This does not provide a solution to the stated problem (as it exists right now). Also, if the user just wished to print the key names, there would still be no need to use a shell loop. – – peak Aug 28 '19 at 17:52
@peak he edited the question after I wrote the answer, originally it just printed them. That was presumably just a simplification of what he really wanted to do with the names. – Barmar Aug 28 '19 at 18:39
Yes, that is why I included the parenthetical remark. – peak Aug 28 '19 at 18:52

peak · Answer 3 · 2019-08-28T17:48:08.337

With the following in program.jq:

# bag of words
def bow(stream): 
  reduce stream as $word ({}; .[($word|tostring)] += 1);

bow(inputs | .host | sub("^.*-availabe";"availabe"))

the invocation:

jq -n -f program.jq input.json

produces

{
  "availabe_zone_A": 2,
  "availabe_zone_B": 1
}

If you really want the output in the STRING --> N format, add -r to the command-line options, and add the following two lines to program.jq:

| to_entries[]
| "\(.key) --> \(.value)"

Comments

You might want to fix the spelling ("available" vs "availabe")
As this example illustrates, jq can handle an input stream consisting of JSON entities -- i.e. a stream of JSON.

Loop through json variable

3 Answers3

Comments