Using Json.Net to parse two semantically identical hierarchy documents and failing

Question

I have two JSON documents which represent the same hierarchy structure and content. The only difference that I can see between the two documents is that the ordering of key value pairs are different. One document parses as I expect and the other doesn't.

I am using "Preserve References Handling" so a node should reference its parent. (The variable "hierarchyTwoNode" in the test is the document that is not having its Parent property set). I have included a test (can be found here) to demonstrate this. Here is a simplified version of the working JSON:

{
  "Root": {
    "$id": "1",
    "Id": "1472459628771017730",
    "Type": "cras",
    "Content": {
      "Name": "lorem"
    },
    "Parent": null,
    "Children": [
      {
        "$id": "2",
        "Id": "1472459628812960771",
        "Type": "morbi",
        "Content": {
          "Name": "ipsum dolor"
        },
        "Parent": {
          "$ref": "1"
        }
      }
    ]
  }
}

And the failing JSON:

{
  "Root": {
    "Parent": null,
    "$id": "1",
    "Children": [
      {
        "Parent": {
          "$ref": "1"
        },
        "$id": "2",
        "Content": {
          "Name": "ipsum dolor"
        },
        "Type": "morbi",
        "Id": "1472459628812960771"
      }
    ],
    "Content": {
      "Name": "lorem"
    },
    "Type": "cras",
    "Id": "1472459628771017730"
  }
}

Could someone give me an idea about what is happening?

Its too big I`m afraid. Stackoverflow has a limit of 30,000 characters. I have included the documents in the test, that's why it is so big. I have also asked the same question at https://github.com/JamesNK/Newtonsoft.Json/issues/1257 if you want to see it all in one go. — dezzy, Mar 25 '17 at 10:11

score 2 · Accepted Answer · answered Mar 25 '17 at 11:02

Could someone give me an idea about what is happening?

Essentially, what is happening is that on one of the JSON string, your metadata properties are placed after the first actual property.

all $xxx properties are metadata and must be placed at the beginning of the object / sub-object.

The reason for this limitation is the following :

if we look at the internals of JSON.Net, when performing the metadata lookup, we can see that as soon as we read a property that is not a metadata, we stop the metadata lookup.

I can only make the fair assumption that it is for computational and memory optimizations, should a JSON file be very large.

To make your code works, simply place all properties starting with a $ at the begining of the object and it should work like a charm.

Thank you gentlemen for all your help. I shifted all meta data properties to the top of each object and it now it deserialises correctly. Thanks for that Fabio. I decided not to use the MetadataPropertyHandling as I will potentially be dealing with large documents, but thanks Brian for pointing it out. Nice to know it is there. — dezzy, Mar 26 '17 at 00:20

score 2 · Answer 2 · answered Mar 25 '17 at 17:06

As @Fabio Salvalai correctly deduced, Json.Net normally expects any metadata properties (such as $id or $type) to appear first in each object for best efficiency in deserialization. If the metadata does not appear first, then Json.Net assumes it isn't there. That is why you are getting different results when the properties are reordered.

Fortunately, Json.Net provides a MetadataPropertyHandling setting to allow it to handle this situation. If you set MetadataPropertyHandling to ReadAhead it should solve your problem. Note that this may have an impact on performance if your JSON is large.

JsonSerializerSettings settings = new JsonSerializerSettings
{
    MetadataPropertyHandling = MetadataPropertyHandling.ReadAhead
};

var hierarchyOne = JsonConvert.DeserializeObject<Hierarchy>(HierarchyOne, settings);
var hierarchyTwo = JsonConvert.DeserializeObject<Hierarchy>(HierarchyTwo, settings);

Using Json.Net to parse two semantically identical hierarchy documents and failing

2 Answers2

Linked