Reading JSON from argument

Question

Below is the command that retrieves JSON into a variable:

$ JSON=$(curl http://some-url)
$ echo $JSON
{ "a" : "1", "b" : "2", "c" : "3" }
$

Need to install jq on docker container. awk is available

Can be parsed only using jq, but

How to read the value of "a" & "b" using jq approach & awk approach?

If you need a tool that's already installed on your servers, instead of `awk`, consider `python`; unlike awk, Python ships with a standard-compliant JSON parser. We already have Q&A entries in the knowledgebase showing how to call Python from bash to extract JSON values. — Charles Duffy, Jul 16 '19 at 20:43
`{ IFS= read -r -d '' a && IFS= read -r -d '' b && IFS= read -r -d '' c; } < <(python -c $'import sys, json\njson_in = json.load(sys.stdin)\nfor key in sys.argv[1:]:\n\tsys.stdout.write(str(json_in.get(key)))\n\tsys.stdout.write("\\0")' a b c <<<'{"a": 1, "b": 2, "c": 3}'); declare -p a b c`, as an example of using Python for this (in a way that works right even if your values contain spaces, newlines, or other surprising content, and runs `python` only once, no matter how many variables you want to extract). — Charles Duffy, Jul 16 '19 at 20:50
@CharlesDuffy Am unable to capture the value in `LHS` in this command `LHS =$(echo "$JSON" | python -c "import sys, json; print json.load(sys.stdin)['a']")` did not work. `echo "$JSON"` works fine — overexchange, Jul 16 '19 at 20:53
...among other problems, adding a space before the `=` makes it no longer an assignment. But really, if you want me to support something, *test the exact code I give you*, not something else. — Charles Duffy, Jul 16 '19 at 20:58
...a more direct translation of the code I gave to only extract `a` from a variable `JSON` and store it in `lhs` might be `IFS= read -r -d '' lhs < <(python -c $'import sys, json\njson_in = json.load(sys.stdin)\nfor key in sys.argv[1:]:\n\tsys.stdout.write(str(json_in.get(key)))\n\tsys.stdout.write("\\0")' a <<<"$JSON")` -- note `lhs` being lowercase; that's because all-caps names are reserved by POSIX-standardized convention for names meaningful to the shell and operating system. — Charles Duffy, Jul 16 '19 at 21:00
See https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html, fourth paragraph, keeping in mind that shell and environment variables share a single namespace. — Charles Duffy, Jul 16 '19 at 21:01
...using NUL-delimited output lets you extend it later to extract multiple variables with only one run, and also prevents variables whose values end in newline literals from getting munged, as they are with the original command substitution approach. — Charles Duffy, Jul 16 '19 at 21:02

Ortomala Lokni · Answer 1 · 2019-07-16T20:42:41.553

1

jq is more appropriate because your data is in JSON format.

To extract the values a and b use:

JSON=$(curl http://some-url)
echo "$JSON" | jq '.a'
echo "$JSON" | jq '.b'

edited Jul 16 '19 at 20:42

answered Jul 16 '19 at 20:36

Ortomala Lokni

56,620
24
188
240

I want to read it from bash variable. Running curl commands twice is not good. I have 10 different keys to read from JSON. – overexchange Jul 16 '19 at 20:37
I've modified my answer. – Ortomala Lokni Jul 16 '19 at 20:39
Can you also help me with `awk`? because `jq` install may need a process request, which takes time – overexchange Jul 16 '19 at 20:39
1

`echo $JSON` is inherently buggy -- always quote: `echo "$JSON"`, or even better, `printf '%s\n' "$JSON"` -- see [BashPitfalls #14](http://mywiki.wooledge.org/BashPitfalls#echo_.24foo), and the APPLICATION USAGE section of [the POSIX spec for `echo`](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html). – Charles Duffy Jul 16 '19 at 20:41
@CharlesDuffy Thanks, answer modified. – Ortomala Lokni Jul 16 '19 at 20:43
1

@overexchange Using awk will make the result dependent on the JSON formatting (e.g. presence and positions of new lines, order of properties). To be robust you will have to implement a JSON parser in awk which is probably outside the scope of the question. – Ortomala Lokni Jul 16 '19 at 20:47

Reading JSON from argument

1 Answers1