Shell Script: How to add json objects to a json output without using jq or any other tool

Question

I am getting below output after executing a script, which I store in a variable

{
  "VariaB": "DONE",
  "VariaC": "DONE",
  "VariaD": null,
  "VariaE": true
}

I have another variable VariaA="ABCD" & VariaF which contains value true or false and want to insert in variable value. I want to print the final variable in below format

{
  "VariaA": "ABCD",
  "VariaB": "DONE",
  "VariaC": "DONE",
  "VariaD": null,
  "VariaE": true,
  "VariaF": false
}

possible duplicate: https://stackoverflow.com/questions/1955505/parsing-json-with-unix-tools — Ente, Sep 03 '19 at 08:03
Is VariaA always a plain string? i.e. no array, object, number or boolean? — Aaron, Sep 03 '19 at 09:47
If your production servers manipulates JSON data, they must have some JSON aware tools in some way or another. Be it Python, PHP, NodeJS..., then maybe Bash/shell is not the right choice, because Bash alone or standard POSIX shell commands are not designated to manipulate JSON, unless you install specialized command-line tools like jq. Either use the others already available JSON aware tools on your production servers, or just add jq as a requirement in the deployment specs of your program. — Léa Gris, Sep 03 '19 at 10:55

nbari · Answer 1 · 2019-09-03T07:46:32.613

1

Looks yout output is JSON, for appending to your output object you could use jq for example try this:

cat json.txt | jq --arg VariaA  ABCD '. + {VariaA: $VariaA}'

In this case, if json.txt contains your input:

{
  "VariaB": "DONE",
  "VariaC": "DONE",
  "VariaD": null,
  "VariaE": true
}

By using jq --arg VariaA ABCD '. + {VariaA: $VariaA}' it will then output:

{
  "VariaB": "DONE",
  "VariaC": "DONE",
  "VariaD": null,
  "VariaE": true,
  "VariaA": "ABCD"
}

If you would like to use more variables you need to use --arg multiple times, for example:

 jq --arg VariaA ABCD --argjson VariaX true '. + {VariaA: $VariaA, VariaX: $VariaX}'

The output will be:

{
  "VariaB": "DONE",
  "VariaC": "DONE",
  "VariaD": null,
  "VariaE": true,
  "VariaA": "ABCD",
  "VariaX": "true"
}

In this example cat json.txt simulates your command output but worth mention that if you wanted to process an existing file you could use (notice the <):

jq --arg VariaA  ABCD '. + {VariaA: $VariaA}' < file.txt

By doing this you do all in one single process.

edited Sep 03 '19 at 07:46

answered Sep 03 '19 at 07:16

nbari

25,603
10
76
131

thanks for the solution, can we do this without jq as its not there in our prod env. – LOrD_ARaGOrN Sep 03 '19 at 07:18
1

I would highly suggest using `jq` or another JSON processor – nbari Sep 03 '19 at 07:23
you are correct this looks very clean but the constraint doesnt allow me to use it. I just checked its not there on prod servers. We have thousands of prod server so practically impossible to get the approval and then get it installed. – LOrD_ARaGOrN Sep 03 '19 at 07:25
we use a job scheduler which run command directly on a server and provide output, its is having word limitation as well thats why i cant write a ling script and push n execute.so, i need to find solution in minimum words. I cant break json output and export each variable and then print accordingly. somehow i need to directly insert the new object. – LOrD_ARaGOrN Sep 03 '19 at 07:31
however i like your solution, thumbs up for that – LOrD_ARaGOrN Sep 03 '19 at 07:33
1

Use `--argjson VariaX true` to have the variable as boolean instead of string – oliv Sep 03 '19 at 07:44
@oliv i want solution without using jq as jq is not there in my prod env. – LOrD_ARaGOrN Sep 03 '19 at 09:04
Just a nitpick, `jq --flags pipeline file` should generally be prefered to `cat file | jq --flags pipeline` – Aaron Sep 03 '19 at 10:14
1

Your production environment is broken if it requires using JSON without the tools to do so properly. – chepner Sep 03 '19 at 13:56

score 1 · Accepted Answer · answered Sep 03 '19 at 10:06

As others said, please do your best to use a JSON-aware tool rather than basic string manipulation, it would be bound to save yourself some effort in the future if not some trouble.

Since you said you currently can't, here's a string manipulation "solution" :

printf "{
  \"VariaA\": \"$VariaA\",
%s
  \"VariaF\": $VariaF
}" "$(grep -v '[{}]' <<< "$input")"

printf handles the whole structure, and takes as parameter the middle block that is from your input. To get that middle block, we use grep to exclude the lines that contain brackets.

Note that this will fail in a lot of cases such as the input not being formatted as usual (a one liner would be proper JSON, but would make that script fail) or suddenly containing nested objects, the variables containing double-quotes, etc.

You can try it here.

score 1 · Answer 3 · answered Sep 03 '19 at 12:53

1

This pipeline does what you need

echo "{"`echo $VariaA | sed "s/{\|}//g"`,`echo "   "$VariaF | sed "s/{\|}//g"`"}" | sed "s/,/,\n/g" | sed "s/{/{\n/" | sed "s/}/\n}/" | uniq

answered Sep 03 '19 at 12:53

Francesco Gasparetto

1,819
16
20

Thank you for this pipeline, but unfortunately it does not work if VariaF's value is a list. This is my case... – Egor Feb 08 '22 at 08:01
1

The user asks help to find a solution a for a specific case. All we can do is to help giving solutions that work in that case, of course they are not universal – Francesco Gasparetto Feb 08 '22 at 11:16

Shell Script: How to add json objects to a json output without using jq or any other tool

3 Answers3