Out of memory for JObject

Question

I have a problem when I try to parse a large json file, which has around 200mb. I'm doing it with Newtonsoft.Json. It gives OutOfMemory exception.

This is my code:

using (StreamReader sr=File.OpenText("path"))
        {
            JObject file= (JObject)JToken.ReadFrom(new JsonTextReader(sr));
        }

How can I do this ? ( preferable using JObject )

score 4 · Accepted Answer · edited May 23 '17 at 12:04

4

You can use JsonTextReader to read text in a DataReader fashion as stated in this question:

Incremental JSON Parsing in C#

You will have to code your own logic to process JSON data, but it will for sure solve your memory issues:

using (var reader = new JsonTextReader(File.OpenText("path")))
{
    while (reader.Read())
    {
        // Your logic here (anything you need is in [reader] object), for instance:
        if (reader.TokenType == JsonToken.StartArray)
        {
            // Process array
            MyMethodToProcessArray(reader);
        }
        else if (reader.TokenType == JsonToken.StartObject)
        {
            // Process object
            MyMethodToProcessObject(reader);
        }
    }
}

You would actually build a recursive JSON parser.

edited May 23 '17 at 12:04

Community

1
1

answered Jan 16 '13 at 10:56

German Latorre

10,058
14
48
59

it seems var property=jsonReader.TokenType.PropertyName; doesn't work. It says 'PropertyName cannot be accessed with an instance reference; qualify it with a type name instead' – Cosmin Jan 16 '13 at 11:03
1

I updated the answer adding some sample code. Using JsonTextReader is like creating a parser from scratch (quite an amount of work if your JSON structure is complex) but, if you feel comfortable with that, it will allow you to avoid your memory issues. – German Latorre Jan 16 '13 at 11:29
Thank you. But how can I acces fields from that file ? For smaller json files I would have done it like this: I would have created an JObject and iterate through it and I could have accessed elements like this item["field_name"].Value(). How can I do that with reader? – Cosmin Jan 16 '13 at 11:41
2

When `reader.TokenType` is `JsonToken.PropertyName`, that means that you just read the name of the JSON field (or property). You can access the name of the property in `reader.Value`. Next call to `reader.Read()` will read the value of that property, which will be placed in `reader.Value` as well. It's easy but, as I told you, it requires some amount of code. – German Latorre Jan 16 '13 at 12:31

Out of memory for JObject

1 Answers1