Serialize JSON array from a web request

Question

I am making what should be a very simple JSON request from a script task from SSIS. The URL return is very large, 20 million characters.

The first URL return line looks like this:

[{"brandId":"9","season":"SG","year":"2003","bomNumber":"00011111","costId":"00001","bomCCNumber":"0000008","firstCost":1.01,"landedFactor":1.234,"elc":9.876,"market":"MA","channel":"CH","destinationCountry":"DC"},...

I've run the URL return through a parser and verified that it is well-formed.

Here is my code.

[DataContract]
public class CostingNegotiated
{

    [DataMember(Name = "brandId")]
    public string brandId { get; set; }

    [DataMember(Name = "season")]
    public string season { get; set; }

    [DataMember(Name = "year")]
    public string year { get; set; }

    [DataMember(Name = "bomNumber")]
    public string bomNumber { get; set; }

    [DataMember(Name = "costId")]
    public string CostID { get; set; }

    [DataMember(Name = "bomCCNumber")]
    public string bomCCNumber { get; set; }

    [DataMember(Name = "firstCost")]
    public string firstCost { get; set; }
    [DataMember(Name = "landedFactor")]
    public double landedFactor { get; set; }

    [DataMember(Name = "elc")]
    public double elc { get; set; }

    [DataMember(Name = "market")]
    public string market { get; set; }

    [DataMember(Name = "channel")]
    public string channel { get; set; }

    [DataMember(Name = "destinationCountry")]
    public string destinationCountry { get; set; }

}

[DataContract]
public class RootObject
{
    [DataMember(Name = "CostingNegotiatedList")]
    public List<CostingNegotiated> CostingNegotiatedList { get; set; }
}

private RootObject GetWebServiceResult(string wUrl)
{

    HttpWebRequest httpWReq = (HttpWebRequest)WebRequest.Create(wUrl);
    httpWReq.Method = "GET";
    httpWReq.ContentType = "application/json";
    httpWReq.Timeout = 300000;
    HttpWebResponse httpWResp = (HttpWebResponse)httpWReq.GetResponse();
    RootObject jsonResponse = null;

    try
    {
        //Get the stream of JSON
        Stream responseStream = httpWResp.GetResponseStream();

        //Deserialize the JSON stream
        using (StreamReader reader = new StreamReader(responseStream))
        {
            string r = reader.ReadToEnd();

            //Deserialize our JSON
            DataContractJsonSerializer sr = new DataContractJsonSerializer(typeof(RootObject));
            MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(r));
            jsonResponse = (RootObject)sr.ReadObject(ms);
        }
    }
    //Output JSON parsing error
    catch (Exception e)
    {
        FailComponent(e.ToString());
    }
    return jsonResponse;

}

I've verified that the string variable r has the JSON string. When I get to this line:

jsonResponse = (RootObject)sr.ReadObject(ms);

The jsonResponse has a RootObject, but the CostingNegotiatedList list variable is NULL. What do I need to do to get this list to populate?

When you say you've checked the URL return, you've ensured you aren't running into length problems as outlined in these articles? http://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers, http://stackoverflow.com/questions/2659952/maximum-length-of-http-get-request — MikeDub, Nov 30 '16 at 01:02
So my actual URL request is pretty short. The RETURN VALUE is extremely long, and I am not 100% sure it's not causing problems. I'm not very familiar with JSON or C#. However, I'm not getting any exceptions on the code. It's just not loading my list object. — Quicksilver, Nov 30 '16 at 01:07
Being a list, do you need to instantiate a new instance of it before values can be set to it? Another possible scenario is it might need assistance getting from an array to a list. — MikeDub, Nov 30 '16 at 01:08
I ... don't think so? I took a sample off another page, and I referred to this SO question to check my syntax: http://stackoverflow.com/questions/13884062/datacontractjsonserializer-returning-null-object?rq=1 — Quicksilver, Nov 30 '16 at 01:13

score 1 · Answer 1 · answered Nov 30 '16 at 02:03

I figured it out. My JSON doesn't have a header element, it's just a straight list, so I don't need my RootObject variable. I changed my code to refer to a list of my objects:

private List GetWebServiceResult(string wUrl) {

HttpWebRequest httpWReq = (HttpWebRequest)WebRequest.Create(wUrl);
httpWReq.Method = "GET";
httpWReq.ContentType = "application/json";
httpWReq.Timeout = 300000;
HttpWebResponse httpWResp = (HttpWebResponse)httpWReq.GetResponse();
List<CostingNegotiated> jsonResponse = null;

try
{
    //Get the stream of JSON
    Stream responseStream = httpWResp.GetResponseStream();

    //Deserialize the JSON stream
    using (StreamReader reader = new StreamReader(responseStream))
    {
        string r = reader.ReadToEnd();

        //Deserialize our JSON
        DataContractJsonSerializer sr = new DataContractJsonSerializer(typeof(List<CostingNegotiated>));
        MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(r));
        jsonResponse = (List<CostingNegotiated>)sr.ReadObject(ms);
    }
}
//Output JSON parsing error
catch (Exception e)
{
    FailComponent(e.ToString());
}
return jsonResponse;

One improvement you could make-- you seem to be reading the response stream using a `StreamReader`, then putting it back into a `MemoryStream` only to read it again with the serializer. You shouldn't need to do that-- just give the responseStream directly to the serializer: `sr.ReadObject(responseStream);` — Brian Rogers, Nov 30 '16 at 02:08

Serialize JSON array from a web request

1 Answers1

Linked