recursively flatten a map golang

Question

Using the below json Im trying to flatten it for ease of accessibility.

Example having 2 resource there can be n number of in this structure

"config": {
    "type": "r1",
    "properties": {
        "p1": "10",
        "p2": "10"
    },
    "connected": [
        {
            "type": "r3",
            "properties": {
              "p1": "10",
              "p2": "10"
            },
            "connected": [
                {}
        },
    ],
}

Custom flatterning logic

func keyValuePairs(m interface{}) map[string]interface{} {
    kvs := make(map[string]interface{})
    if reflect.ValueOf(m).Kind() == reflect.Map {
        mp, ok := m.(map[string]interface{})
        if ok {
            var key string
            var value interface{}
            for k, v := range mp {
                switch k {
                case "type":
                    key = v.(string)
                case "properties":
                    value = v
                case "connected":
                    if collection, ok := v.([]interface{}); ok {
                        for _, c := range collection {
                            for nk, nv := range keyValuePairs(c) {
                                nnv, _ := nv.(map[string]interface{})
                                _, ok := nnv["connectedTo"]
                                if !ok {
                                   nnv["connectedTo"] = key
                                }
                                kvs[nk] = nv
                            }
                        }
                    }
                default:
                    for nk, nv := range keyValuePairs(v) {
                        kvs[nk] = nv
                    }
                }
            }
            if key != "" {
                kvs[key] = value
            }
        } else {
            for k, v := range m.(map[string]interface{}) {
                kvs[k] = v
            }
        }
    }
    return kvs
}

The desired output is kinda fluctuationg, some runs I get the output what I require and some executions "connectedTo" attribute is empty.

{
"r1": {
        "p1": "10",
        "p2": "10"
    },
"r3" : {
        "connectedTo": "r1",
        "p1": "10",
        "p2": "10"
    },
}

I think the executions are not sequential. Please do correct me if Im wrong.

while recursively calling the function sometimes the order is not maintained. Sometimes i observe order as 1,2,3 sometimes its 1,3,2 — user8866279, Jun 02 '22 at 10:54
maps in go are not ordered. Iterating their keys is pesudo random, if you mean that. — The Fool, Jun 02 '22 at 10:57
Okay. But do you have idea why the results is not proper on some executions? — user8866279, Jun 02 '22 at 10:58
I dont really get what your code is doing, but I havent really looked at it. I am just saying if you rely on the order of keys, than that is a bug. You cannot rely on the order. Apart from that, its too much relfection for my taste, if your object has a well known structure, you shouldn't need reflection at all, you know what key to pick. If it has connections, you call the function again. — The Fool, Jun 02 '22 at 11:00
I wish there was defined struct. Its actually dynamically based on the json we create the resource so "r1" can be top level and also into 2nd or 3rd level. So i thought flatterning the struct will help irrespective of the nested resource — user8866279, Jun 02 '22 at 11:05

icza · Accepted Answer · 2022-06-02T11:27:15.233

Iteration order over a map is random. For details, see Why are iterations over maps random? and How to iterate maps in insertion order?

Now look at your loop:

var key string
var value interface{}
for k, v := range mp {
    switch k {
    case "type":
        key = v.(string)
    case "properties":
        value = v
    case "connected":
        if collection, ok := v.([]interface{}); ok {
            for _, c := range collection {
                for nk, nv := range keyValuePairs(c) {
                    nnv, _ := nv.(map[string]interface{})
                    _, ok := nnv["connectedTo"]
                    if !ok {
                       nnv["connectedTo"] = key
                    }
                    kvs[nk] = nv
                }
            }
        }
    default:
        for nk, nv := range keyValuePairs(v) {
            kvs[nk] = nv
        }
    }
}

You are using the key variable in the "connected" branch, but the order in which "type" and "connected" will be reached is non-deterministic (random).

The "type" branch is what sets key, but if "connected" is reached first, key will be empty.

You must not rely on map iteration order.

An easy fix is to first get the value associated with"type" first and assign it to key (which you use in the "connected" branch), before the loop.

For example:

key, _ := mp["type"].(string)
value := mp["properties"]
// Now process other properties:
for k, v := range mp {
    switch k {
    case "type", "properties": // Already handled
    case "connected":
        if collection, ok := v.([]interface{}); ok {
            for _, c := range collection {
                for nk, nv := range keyValuePairs(c) {
                    nnv, _ := nv.(map[string]interface{})
                    _, ok := nnv["connectedTo"]
                    if !ok {
                       nnv["connectedTo"] = key
                    }
                    kvs[nk] = nv
                }
            }
        }
    default:
        for nk, nv := range keyValuePairs(v) {
            kvs[nk] = nv
        }
    }
}

Hi @icza, i have updated the logic, mistake from my side I used different code. The logic to assign `connectedTo` is under `connected` and not under `default` — user8866279, Jun 02 '22 at 11:26
@user8866279 I have updated my answer to reflect your edited code, but doesn't change the fact and reasoning. — icza, Jun 02 '22 at 11:27

recursively flatten a map golang

1 Answers1