How to manually flatten Elasticsearch nested JSON documents using Java?

Question

I want to generate some documentation for my Elasticsearch document structure. The problem is that i'm storing nested JSON in my index but i want to document the flattend JSON format¹ that Elasticsearch is generating.

Is there a way to flatten this JSON similar to the way that Elasticsearch is producing using the ES Java API?

If possible i don't want to startup Elasticsearch for this task.

Example JSON:

{
  "title": "Nest eggs",
  "body":  "Making your money work...",
  "tags":  [ "cash", "shares" ],
  "comments": [ 
    {
      "name":    "John Smith",
      "comment": "Great article",
      "age":     28,
      "stars":   4,
      "date":    "2014-09-01"
    },
    {
      "name":    "Alice White",
      "comment": "More like this please",
      "age":     31,
      "stars":   5,
      "date":    "2014-10-22"
    }
  ]
}

The document will look like this once Elasticsearch has flattend it.

{
  "title":            [ eggs, nest ],
  "body":             [ making, money, work, your ],
  "tags":             [ cash, shares ],
  "comments.name":    [ alice, john, smith, white ],
  "comments.comment": [ article, great, like, more, please, this ],
  "comments.age":     [ 28, 31 ],
  "comments.stars":   [ 4, 5 ],
  "comments.date":    [ 2014-09-01, 2014-10-22 ]
}

[1] https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-objects.html

Would [something like this](http://fiddle.jshell.net/blowsie/S2hsS/show/light/) or [this](https://github.com/hughsk/flat) help? Other solutions are [available here](http://stackoverflow.com/questions/19098797/fastest-way-to-flatten-un-flatten-nested-json-objects), too. — Val, Aug 19 '15 at 10:58

Jotschi · Accepted Answer · 2015-09-21T15:14:24.980

4

I wrote my own algorithm that flattens the Map which would be used to create the JSON.

private void flatten(Map<String, Object> map, Map<String, Object> output, String key) throws JSONException {
        String prefix = "";
        if (key != null) {
            prefix = key + ".";
        }
        for (Entry<String, Object> entry : map.entrySet()) {
            String currentKey = prefix + entry.getKey();
            if (entry.getValue() instanceof Map) {
                flatten((Map<String, Object>) entry.getValue(), output, prefix + entry.getKey());
            } else if (entry.getValue() instanceof List) {
                output.put(currentKey, entry.getValue());
            } else {
                output.put(currentKey, entry.getValue());
            }
        }
    }

Usage example:

    Map<String, Object> outputMap = new TreeMap<>();
    flatten(inputMap, outputMap, null);
    JSONObject json = new JSONObject(outputMap);
    String jsonStr = json.toString(4);

edited Sep 21 '15 at 15:14

answered Sep 14 '15 at 11:14

Jotschi

3,270
3
31
52

Have you tested it? How is it handling collections? – fps Sep 14 '15 at 12:21
My model is only using lists, maps and strings. Lists are preserved and maps are collapsed. I have not yet encountered problems so far. – Jotschi Sep 14 '15 at 12:28
Glad you managed it to work. I asked because I did something very similar a few weeks ago and it was especially tricky to flatten collections of nested objects, when these objects were POJOs. – fps Sep 14 '15 at 12:31

How to manually flatten Elasticsearch nested JSON documents using Java?

1 Answers1