Merge 2 JSON files with jq

Question

I need to write a command via jq which is:

1. compares old and new config json files (old-o-ru.conf and new-o-ru.conf).
2. Copies the value of the element from the old config to the new one, if the new one has this element and their types match.
3. Unaffected elements in the new config should remain.

old.conf:

{
    "ru_conf": {
        "network": {
            "interface": "eth1",
            "mac_address": "FF:FF:FF:FF:FF:FF",
            "setup": true,
            "interfaceы": ["eth0", "eth1"],
            "numbers": [1, 2, 3],
            "float": 1111.1111,
            "partitions": [
                {
                  "dev1": "/mnt/data/",
                  "threshold": 20,
                }
            ]
        }
    }
}

new.conf:

{
    "ru_conf": {
        "network": {
            "interface": "str",
            "mac_address": "str",
            "setup": false,
            "interfaceы": ["str1", "str2"],
            "numbers": [0, 0, 0],
            "int": 0,
            "partitions": [
                {
                  "dev1": "str",
                  "threshold": 0,
                }
            ]
        }
    }
}

Differences in configs: there is no "int" field in old.conf there is no "float" field in new.conf

I am using the command:

jq -s '.[1].ru_conf as $new | .[0] | .ru_conf |= reduce paths(scalars) as $p (.; if ($new | getpath($p)) and (getpath($p) | type) == ($new | getpath($p) | type) then setpath($p; $new | getpath($p)) else . end)' ./new.conf ./old.conf > merged-config.conf

Result:

{
  "ru_conf": {
    "network": {
      "interface": "eth1",
      "mac_address": "FF:FF:FF:FF:FF:FF",
      "setup": false,
      "interfaces": [
        "eth0",
        "eth1"
      ],
      "numbers": [
        1,
        2,
        3
      ],
      "int": 0,
      "partitions": [
        {
          "dev1": "/mnt/data/",
          "threshold": 20
        }
      ]
    }
  }
}

Expected output:

{
    "ru_conf": {
        "network": {
            "interface": "str",
            "mac_address": "str",
            "setup": true,
            "interfaces": ["str1", "str2"],
            "numbers": [0, 0, 0],
            "int": 0,
            "partitions": [
                {
                  "dev1": "str",
                  "threshold": 0
                }
            ]
        }
    }
}

Differences from result and expected result: "setup": false

Also tried this:

jq \
 --argjson a "$(cat ./new.conf)" \
 --argjson b "$(cat ./old.conf)" \
 -n '
   $a | .ru_conf | keys_unsorted as $whitelist 
 | $b | .ru_conf | with_entries( select(.key | IN($whitelist[])) ) as $update
 | $a | .ru_conf += $update '

Result:

{
  "ru_conf": {
    "network": {
      "interface": "eth1",
      "mac_address": "FF:FF:FF:FF:FF:FF",
      "setup": true,
      "interfaces": [
        "eth0",
        "eth1"
      ],
      "numbers": [
        1,
        2,
        3
      ],
      "float": 1111.1111,
      "partitions": [
        {
          "dev1": "/mnt/data/",
          "threshold": 20
        }
      ]
    }
  }
}

But the second command does not support type comparison and nesting

How can I achieve the required result?

Answer 1

Solution in TXR Lisp:

$ txr merge-conf.tl old.conf new.conf
{
  "ru_conf" : {
    "network" : {
      "int" : 0,
      "partitions" : [
        {
          "threshold" : 0,
          "dev1" : "str"
        }
      ],
      "mac_address" : "str",
      "interface" : "str",
      "setup" : true,
      "numbers" : [
        0,
        0,
        0
      ],
      "interfaceы" : [
        "str1",
        "str2"
      ]
    }
  }
}

Note that this solution does not take care of renaming ""interfaceы" to "interfaces". That looks like a typo.

We define a hash-merge function which merges two hash tables using a merge-fun functional argument to decide what to do. It calls merge-fun with four arguments: a keyword which is { :both, :left-only or :right-only }, the hash key being merged and the left and right values.

The function calculates and returns the merged value.

In the :left-only and :right-only cases, if the function returns nil, the object will not appear in the merged hash.

The function funny-merge then does some argument pattern matching to determine what to do. For instance one pattern says that if we have a "float" or "int" element that is on the left side only (old config), then that does not appear. Another rule says that the left side's "setup" is taken, rather than the right.

(defun hash-merge (lhash rhash merge-fun)
  (let ((ohash (copy lhash)))
    (dohash (key val ohash)
      (unless (in rhash key)
        (iflet ((nval [merge-fun :only-left key val nil]))
          (set [ohash key] nval)
          (del [ohash key]))))
    (dohash (key val rhash ohash)
      (if (in ohash key)
        (set [ohash key] [merge-fun :both key [lhash key] val])
        (iflet ((nval [merge-fun :only-right key nil val]))
          (set [ohash key] nval)
          (del [ohash key]))))))

(defun-match funny-merge
  ((@nil @nil @(hashp @left) @(hashp @right)) [hash-merge left right funny-merge])
  ((:both "setup" @left @right) left)
  ((:left-only @(or "float" "int") @nil @nil) nil)
  ((:left-only @nil @left @nil) left)
  ((@(or :both :right-only) @nil @nil @right) right))

(let ((*read-bad-json* t)
      (*print-json-format* :standard))
  (match (@(@oconf (file-get-json)) @(@nconf (file-get-json))) *args*
    (put-jsonl (funny-merge nil nil oconf nconf))))