How can jq read lines from a file and create valid a JSON?

Question

I need help with creating a valid JSON file via jq which read lines from the file.

I have some text.txt file:

rabbit1-test1
rabbit1-test2
rabbit1-test3

And the bash-script:

VHOSTNAME=/
NODENAME=node1

input="test.txt"
while IFS= read -r line; do jq -n \
                  --arg a "$VHOSTNAME" \
                  --arg b "$line"  \
                  --arg c "$NODENAME" \
                  '{"data": [{"{#VHOSTNAME}": $a, "{#QUEUENAME}": $b, "{#NODENAME}": $c}]}' ;done < "$input"

But, when I run the script, the JSON isn't valid:

{
  "data": [
    {
      "{#VHOSTNAME}": "/",
      "{#QUEUENAME}": "rabbit1-test1",
      "{#NODENAME}": "node1"
    }
  ]
}
{
  "data": [
    {
      "{#VHOSTNAME}": "/",
      "{#QUEUENAME}": "rabbit1-test2",
      "{#NODENAME}": "node1"
    }
  ]
}
{
  "data": [
    {
      "{#VHOSTNAME}": "/",
      "{#QUEUENAME}": "rabbit1-test3",
      "{#NODENAME}": "node1"
    }
  ]
}

I can't understand why jq reads file incorrectly. Please help me.

What output do you expect / hope for? This does look like valid JSON fragments; do you mean you want commas between them? — tripleee, Feb 18 '20 at 13:30
This does look like real valid JSON**S**, because you are running `jq` several times and getting several valid JSON. — Poshi, Feb 18 '20 at 13:31
The valid json is { "data": [ { "{#VHOSTNAME}": "/", "{#QUEUENAME}": "rabbit1-test1", "{#NODENAME}": "node1" }, { "{#VHOSTNAME}": "/", "{#QUEUENAME}": "rabbit1-test2", "{#NODENAME}": "node1" } ] } — perrfect, Feb 18 '20 at 13:33

score 1 · Accepted Answer · answered Feb 18 '20 at 13:40

1

Why don't you use jq's split ?

jq -Rs --arg a '/' --arg c 'node1' '[ .[:-1] |
      split("\n")[] |
      { "{#VHOSTNAME}" : $a , 
        "{#QUEUENAME}": . , 
        "{#NODENAME}": $c } ] |
       { "data" : . }' sample.txt

Out:

{
  "data": [
    {
      "{#VHOSTNAME}": "/",
      "{#QUEUENAME}": "rabbit1-test1",
      "{#NODENAME}": "node1"
    },
    {
      "{#VHOSTNAME}": "/",
      "{#QUEUENAME}": "rabbit1-test2",
      "{#NODENAME}": "node1"
    },
    {
      "{#VHOSTNAME}": "/",
      "{#QUEUENAME}": "rabbit1-test3",
      "{#NODENAME}": "node1"
    }
  ]
}

answered Feb 18 '20 at 13:40

Sorin

5,201
2
18
45

But, when i run jq not from file, but using "while", it doesn't work. – perrfect Feb 18 '20 at 14:58
@perrfect why do you need a while ? – Sorin Feb 18 '20 at 15:05
Because I need the script for rabbitMQ queues. And I want to get for command jq, all queues, via command - "rabbitmqctl list_queues", aren't using a .txt file. – perrfect Feb 18 '20 at 15:10
still you shouldn't need a while. `rabbitmqctl list_queues | jq -Rs --arg a '/' --arg c 'node1' ....` – Sorin Feb 18 '20 at 15:13
Sorry, but I need use "read lines". In rabbitMQ i have 1000 queues and i must read the queues blocks of 100 pieces – perrfect Feb 19 '20 at 08:46
@perrfect - I'm sorry, but you can't change the question incrementally and expect people to update their answer every time. Please read https://stackoverflow.com/help/minimal-reproducible-example. You can try to merge several files using https://stackoverflow.com/questions/19529688/how-to-merge-2-json-objects-from-2-files-using-jq – Sorin Feb 19 '20 at 08:56

How can jq read lines from a file and create valid a JSON?

1 Answers1