How do I merge 2 json files?

Question

file1:

{
    "status": {
        "errorCode": 0,
        "recordsTotal": 6,
        "recordsInResponse": 2
    },
    "records": [
        {
            "itemID": 128,
            "name": "foo barn"
        },
        {
            "itemID": 64,
            "name": "fee fom"
        }
    ]
}

file2:

{
    "status": {
        "errorCode": 0,
        "recordsTotal": 6,
        "recordsInResponse": 2
    },
    "records": [
        {
            "itemID": 32,
            "name": "pim pom"
        },
        {
            "itemID": 256,
            "name": "ping pong"
        }
    ]
}

wanted:

"records": [ // records array is not needed, having just items iterated is fine as well.
    {
        "itemID": 128,
        "name": "foo barn"
    },
    {
        "itemID": 64,
        "name": "fee fom"
    },
    {
        "itemID": 32,
        "name": "pim pom"
    },
    {
        "itemID": 256,
        "name": "ping pong"
    }
]

From what I have tried, I have dismissed filtering the records, focused on only combining the files. I'm not very knowlegable with jq. I got the nearest results with:

jq -s '.[0] += .[1]'
jq -s '.[0] |= . + .[1]'

How do I merge 2 json files, probably with jq?

score 2 · Answer 1 · answered Sep 06 '20 at 21:26

Use -s to create one array from both the files, than use .[0] and .[1] to get each file's structure.

jq -s '.[0].records + .[1].records' file1 file2

To include the toplevel key, wrap the expression to an object:

jq -s '{records: (.[0].records + .[1].records)}' file1 file2

score 2 · Accepted Answer · answered Sep 06 '20 at 23:48

2

You can avoid creating an initial array with inputs.

(A) If you just want the objects

jq -n 'inputs | .records[]'

Result:

{
  "itemID": 128,
  "name": "foo barn"
}
{
  "itemID": 64,
  "name": "fee fom"
}
{
  "itemID": 32,
  "name": "pim pom"
}
{
  "itemID": 256,
  "name": "ping pong"
}

(B) If you want the objects in an array

jq -n 'reduce inputs as $in ([]; . += ($in.records))'

Result:

[
  {
    "itemID": 128,
    "name": "foo barn"
  },
  {
    "itemID": 64,
    "name": "fee fom"
  },
  {
    "itemID": 32,
    "name": "pim pom"
  },
  {
    "itemID": 256,
    "name": "ping pong"
  }
]

(C) If you want the objects in an array named "records"

jq -n 'reduce inputs as $in ({}; .["records"] += ($in.records))'

Result:

{
  "records": [
    {
      "itemID": 128,
      "name": "foo barn"
    },
    {
      "itemID": 64,
      "name": "fee fom"
    },
    {
      "itemID": 32,
      "name": "pim pom"
    },
    {
      "itemID": 256,
      "name": "ping pong"
    }
  ]
}

answered Sep 06 '20 at 23:48

user197693

1,935
10
8

I have encountered this with jq before, with A, why are the objects not coma separated? – u15p7fgy863eiq5 Sep 08 '20 at 04:42
Why is the `-n` flag needed? Every other possible solution I have seen uses the `-s` flag instead. "Don't read any input at all!" in the manpages makes it somewhat more confusing, though it mentions it allows to "construct JSON data from scratch". – u15p7fgy863eiq5 Sep 08 '20 at 04:44
(A) isn't json. So maybe the best answer is to turn the question around. Why *would* the objects be comma separated? – user197693 Sep 08 '20 at 21:04
1

Peter Koppstein's ["A Stream oriented Introduction to jq"](https://github.com/pkoppstein/jq/wiki/A-Stream-oriented-Introduction-to-jq#on-the-importance-of-inputs) explains why you might prefer a solution using `inputs` to one that uses `-s`. – user197693 Sep 08 '20 at 21:07
You use `-n` to avoid losing the first entity. – user197693 Sep 08 '20 at 21:09
(A) - Didn't know that, what makes json json? – u15p7fgy863eiq5 Sep 10 '20 at 17:18
(A) has four valid json objects. The collection of all four together is valid json only if they are in an array; then they would be comma separated. What `inputs` is doing, as I understand it, is going json entity by json entity. In (A) it puts out the objects one at a time rather than collecting them. In (B) and (C) we tell jq to collect the objects in arrays. The Koppstein article helps here; the [JSON definition](https://www.json.org/json-en.html) helps too. – user197693 Sep 11 '20 at 18:20

How do I merge 2 json files?

2 Answers2