Extract json value with sed

Question

I have a json result and I would like to extract a string without double quotes

{"value1":5.0,"value2":2.5,"value3":"2019-10-24T15:26:00.000Z","modifier":[]}

With this regex I can extract the value3 (019-10-24T15:26:00.000Z) correctly

sed -e 's/^.*"endTime":"\([^"]*\)".*$/\1/'

How can I extract the "value2" result, a string without double quotes?

I need to do with sed so can’t install jq. That’s my problem

Answer 1

With GNU sed for -E to enable EREs:

$ sed -E 's/.*"value3":"?([^,"]*)"?.*/\1/' file
2019-10-24T15:26:00.000Z

$ sed -E 's/.*"value2":"?([^,"]*)"?.*/\1/' file
2.5

With any POSIX sed:

$ sed 's/.*"value3":"\{0,1\}\([^,"]*\)"\{0,1\}.*/\1/' file
2019-10-24T15:26:00.000Z

$ sed 's/.*"value2":"\{0,1\}\([^,"]*\)"\{0,1\}.*/\1/' file
2.5

The above assumes you never have commas inside quoted strings.

Answer 2

Just run jq a Command-line JSON processor

$ json_data='{"value1":5.0,"value2":2.5,"value3":"2019-10-24T15:26:00.000Z","modifier":[]}'
$ jq '.value2' <(echo "$json_data")
2.5

with the key .value2 to access the value you are interested in.

This link summarize why you should NOT use, regex for parsing json (the same goes for XML/HTML and other data structures that are in theory can be infinitely nested)

Regex for parsing single key: values out of JSON in Javascript

If you do not have jq available:

you can use the following GNU grep command:

$ echo '{"value1":5.0,"value2":2.5,"value3":"2019-10-24T15:26:00.000Z","modifier":[]}' | grep -zoP '"value2":\s*\K[^\s,]*(?=\s*,)'
2.5

using the regex detailed here:

"value2":\s*\K[^\s,]*(?=\s*,)

demo: https://regex101.com/r/82J6Cb/1/

This will even work if the json is not linearized!!!!

With python it is also pretty direct and you should have it installed by default on your machine even if it is not python3 it should work

$ cat data.json 
{"value1":5.0,"value2":2.5,"value3":"2019-10-24T15:26:00.000Z","modifier":[]}
$ cat extract_value2.py 
import json

with open('data.json') as f:
    data = json.load(f)
    print(data["value2"])
$ python extract_value2.py 
2.5

Answer 3

You can try this :

creds=$(eval aws secretsmanager get-secret-value --region us-east-1 --secret-id  dpi/dev/hivemetastore --query SecretString --output text )
passwd=$(/bin/echo "${creds}" | /bin/sed -n 's/.*"password":"\(.*\)",/\1/p' | awk -F"\"" '{print $1}')

it is definitely possible to remove the AWK part though ...

Answer 4

if your data in 'd' file, try gnu sed

sed -E 's/[{,]"\w+":([^,"]+)/\1\n/g ;s/(.*\n).*".*\n/\1/' d

Answer 5

To extract all values in proper list form to a file using sed(LINUX).

sed 's/["{}\]//g' <your_file.json> | sed 's/,/\n/g' >> <your_new_file_to_save>

• sed 's/regexp/replacement/g' inputFileName > outputFileName
  

• In some versions of sed, the expression must be preceded by -e to indicate that an expression follows.
• The s stands for substitute, while the g stands for global, which means that all matching occurrences in the line would be replaced. I've put [ ] inside it as elements that you wanna remove from .json file.
• The pipe character | is used to connect the output from one command to the input of another. Then, the last thing I did is substitute , and add a \n, known as line breaker.

If you want to show a single value see below command:

sed 's/["{}\]//g' <your_file.json> | sed 's/,/\n/g' | sed 's/<ur_value>//p'

• p is run; this is equivalent to /pattern match/! p as per above; i.e., "if the line does not match /pattern match/, print it". So the complete command prints all the lines from the first occurrence of the pattern to the last line, but suppresses the ones that match.