jq how to add an element to multiple arrays

Question

I have an issue with such json file:

{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16132421/",
  "id": "16132421",
  "gender": "W"
}
{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16104273/",
  "id": "16104273",
  "gender": "W"
}
{}
{
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}

I has 3 arrays. I would like to replicate the last array to each of the prevoius ones, so it will look like this:

{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16104259/",
  "id": "16104259",
  "gender": "W",
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}
{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16132421/",
  "id": "16132421",
  "gender": "W",
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}

I tried to just add or multiply the arrays, but it doesn't work.

When I add specific keys ( {name, url, id, gender} + {brand, cats} ) it displays the key's in place, but it sets values as null.

Can you help me find the solution?

Note that StackSnippets are only for HTML/CSS/JavaScript -- things that can be run in a browser. Use the `{}` button, not the snippet button, for syntax-highlighting other languages. — Charles Duffy, Oct 14 '22 at 20:26

pmf · Answer 1 · 2022-10-14T20:34:53.417

Convert the stream into an array either by using -n with [inputs], or by using -s, then add the last item to all but the last .[:-1][]. To filter out the empty object, select by comparing it to {} or by checking its length.

jq -n '[inputs] | (.[:-1][] | select(. != {})) + last'

Demo

jq -s '(.[:-1][] | select(length > 0)) + last'

Demo

Output:

{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16132421/",
  "id": "16132421",
  "gender": "W",
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}
{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16104273/",
  "id": "16104273",
  "gender": "W",
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}

`jq -n '[inputs] | …'` is equivalent to `jq -s '…', right? The command becomes shorter (and simpler?). ([Difference between slurp, null input, and inputs filter](https://stackoverflow.com/q/73843868/112968)) — knittl, Oct 14 '22 at 20:22
@knittl Yes, I was about to add it but your comment was faster :) — pmf, Oct 14 '22 at 20:24

score 0 · Answer 2 · answered Oct 14 '22 at 21:01

If the stream does not fit into your memory, you have to use the --stream option which breaks down the input into manageable pieces. With streamed processing, however, you cannot easily access different parts at the same time. Therefore, you'd need one pass to retrieve the last item, and another one to add it to the other ones. As during processing there is no indicator for an item to being the last one, you could instead filter out those that entirely match the one being added (which conveniently happens to also filter out previously empty items).

jq --argfile last <(jq 'reduce inputs as $in (.; $in)' big.json) --stream \
  -n 'fromstream(inputs) + $last | select(. != $last)' big.json

{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16132421/",
  "id": "16132421",
  "gender": "W",
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}
{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16104273/",
  "id": "16104273",
  "gender": "W",
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}

jq how to add an element to multiple arrays

2 Answers2