0

I have a script in my application that gathers some JSON data from my PHP REST API. The API uses json_encode() to serialize a multidimensional array that it has generated. The script within my app should deserialize this into the equivalent in C# so that I can loop through it as each sub-array represents a "row" of information from the database. However, I am presented with this error:

Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: A. Path '', line 0, position 0.
  at Newtonsoft.Json.JsonTextReader.ParseValue () [0x002b3] in <2073514815234917a5e8f91b0b239405>:0 
  at Newtonsoft.Json.JsonTextReader.Read () [0x0004c] in <2073514815234917a5e8f91b0b239405>:0 
  at Newtonsoft.Json.JsonReader.ReadAndMoveToContent () [0x00000] in <2073514815234917a5e8f91b0b239405>:0 
  at Newtonsoft.Json.JsonReader.ReadForType (Newtonsoft.Json.Serialization.JsonContract contract, System.Boolean hasConverter) [0x0004a] in <2073514815234917a5e8f91b0b239405>:0 
  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType, System.Boolean checkAdditionalContent) [0x000db] in <2073514815234917a5e8f91b0b239405>:0 
  at Newtonsoft.Json.JsonSerializer.DeserializeInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x

The JSON (exactly as my app receives it from the server)

[{"type":"0","ID":"1","groupID":"0","authorID":"14","contents":"rfgwgfgdfssd","time_stamp":"0-0-0"}, {"type":"0","ID":"2","groupID":"0","authorID":"14","contents":"whwrgthwr","time_stamp":"0-0-0"}]

The Script

async void getUpdatesAsync()
        {
            Console.WriteLine("starting...");

            BackendConnect connectionHandler = new BackendConnect("vocal/posts/index.php");

            var updatesRequestInfo = new Dictionary<string, string>
            {

                {"sessionID", "c1f7a973c04e18a05342ecc2d16f7339"},
                {"type", "updateFeed"},
                {"userID", "14"}

            };

            connectionHandler.addData(updatesRequestInfo);

            string updatesReturn = await connectionHandler.sendForm(); // returns the json

            Console.WriteLine(updatesReturn); // the above json was copied and pasted from this output

            List<Dictionary<string, string>> updatesArray = JsonConvert.DeserializeObject<List<Dictionary<string, string>>>(updatesReturn); // this line causes error

            Console.WriteLine(updatesArray);

            //updatesArray.ForEach(Console.WriteLine);

        }

Another script in my app that also uses JsonConvert.DeserializeObject<DataType>(stringToDeserialize) works when a single dimensional array (["SUCCESS", "adlhfbaf"]) is given to the deserialize function and is able to parse this into a simple list, but fails when trying with the multidimensional array JSON above.

So far I have tried setting the updatesArray variable to an endless amount of different data types to see if that was the problem which I suspect it is, but as a newbie to languages like C# where you have to pre-define datatypes, I'm having a very hard time solving this one.

Essentially, I would like to import it as a list of dictionaries in C# so that I can use foreach() to loop through each index of the initial list and recover a dictionary where I can grab specific values (such as contents). Below is an example of what i'm looking for it to import (incorrect syntax I know but a rough idea at least)

var updatesArray = new List<Dictionary<string, string>>
                {
                    {
                        {"type", "0"},
                        {"ID", "1"},
                        {"groupID", "0"},
                        {"authorID", "14"},
                        {"contents", "rfgwgfgdfssd"},
                        {"time_stamp", "0-0-0"}
                    },
                    {
                        {"type", "0"},
                        {"ID", "2"},
                        {"groupID", "0"},
                        {"authorID", "14"},
                        {"contents", "whwrgthwr"},
                        {"time_stamp", "0-0-0"}
                    }

                };
Matthew Swallow
  • 179
  • 1
  • 11
  • 2
    can you parse your json into a JObject? – Jason Dec 21 '20 at 14:07
  • Hi, I'm not familiar with JObjects, how would this work? – Matthew Swallow Dec 21 '20 at 14:34
  • 1
    Instead of deserializing into a dictionary list. Create a model for your response and deserialize it into a List of that model. This will also give you IntelliSense to your data in the code – ejwill Dec 21 '20 at 14:48
  • 1
    I've tested `JsonConvert.Deserialize>` for your string and it works. Either your string is actually different or your `List` or `Dictionary` is not from `System.Collections.Generic` namespace. Also, what Newtonsoft.Json version do you use? – Quercus Dec 21 '20 at 14:48
  • Can't reproduce either, see https://dotnetfiddle.net/PdebHw. Incidentally, that is not a multidimensional array, it is a 1d array of objects. – dbc Dec 21 '20 at 14:59
  • Your JSON string might have a [BOM](https://en.wikipedia.org/wiki/Byte_order_mark) at the beginning. A BOM shouldn't appear in a string, it should be processed and removed during decoding as explained in [Encoding.UTF8.GetString doesn't take into account the Preamble/BOM](https://stackoverflow.com/q/11701341/3744182) and [How do I ignore the UTF-8 Byte Order Marker in String comparisons?](https://stackoverflow.com/a/2915239/3744182). But if it somehow did, you can strip as shown in [Strip Byte Order Mark from string in C#](https://stackoverflow.com/q/1317700/3744182). – dbc Dec 21 '20 at 15:13
  • 1
    `JObject.Parse(jsonstring);` – Jason Dec 21 '20 at 15:21
  • Hi everyone, thank you for your responses. @Quercus and @dbc I have copied and pasted the JSON output from the `Console.WriteLine` that I commented next to in the script. I am unaware on how I can see if the raw format isn't right. The part returned by `sendForm()` comes from the line `return await responseMessage.Content.ReadAsStringAsync();` in another file which is from the `System.Net.Http` library so I'm not sure how formatting could have been damaged. The URL of the REST script is: https://destenica.co.uk/vocal/posts/index.php if you wish to try and identically replicate my error – Matthew Swallow Dec 21 '20 at 15:47
  • @Jason Using `JObject.Parse` yields an identical error – Matthew Swallow Dec 21 '20 at 15:53
  • "how I can see if the raw format isn't right" - use the debugger – Jason Dec 21 '20 at 15:54
  • @Jason I have used `Console.WriteLine` and looked in the debugger's console window. Is this what you mean? Sorry, I'm quite new to Visual Studio – Matthew Swallow Dec 21 '20 at 15:58
  • examine the variable and it should allow you to view it in Hex. Or try just checking the string length and compare it to what you expect. Or just assume it has a BOM and take the steps suggested in the link @dbc posted – Jason Dec 21 '20 at 16:02
  • @Jason @dbc I have tried removing a byte order mark with no luck. On my way however I tested the length of the string, which I expected to be '196'. However `String.Length` returns '629' so there are a lot of hidden characters. Any ideas? I am not in a position where I am able to majorly change the code, just need a quick line or two to re-format or remove characters from this string. (it is a System.String at runtime just in case you want to know, I've checked this) – Matthew Swallow Dec 21 '20 at 21:20
  • again, look at your data in the Hex editor. Or write it to a file. Or covert it to an array and output the console if that's the only method you're familiar with. Or use Postman to examine the data coming from the server. – Jason Dec 21 '20 at 21:26
  • the url you supplied returns an error. Does it require parameters? Is it a Post or a Get, what are the required headers, etc? – Jason Dec 21 '20 at 21:40

2 Answers2

1

Instead of deserializing into a dictionary. Create a model for your response. Then deserialize it to a List of that model. You can see an example of this below. I based it on the code you supplied just replaced all of the call outs with a variable that holds a JSON string as you would get from your callout

public static void Main()
{

    {       
        //json string
        //you should fix the timestamp coming back...not a real timestamp
        var testresponse = "[{\"type\":\"0\",\"ID\":\"1\",\"groupID\":\"0\",\"authorID\":\"14\",\"contents\":\"rfgwgfgdfssd\",\"time_stamp\":\"0-0-0\"},{\"type\":\"0\",\"ID\":\"2\",\"groupID\":\"0\",\"authorID\":\"14\",\"contents\":\"whwrgthwr\",\"time_stamp\":\"0-0-0\"}]";

        Console.WriteLine(testresponse);

        var updatesArray = JsonConvert.DeserializeObject<List<TestModel>>(testresponse);

        Console.WriteLine(updatesArray[0].Id);
        Console.WriteLine(updatesArray[1].Id);
        
        var after = JsonConvert.SerializeObject(updatesArray);
        
        Console.WriteLine(after);

    }
}

If the timestamp is a DateTime set the data type to DateTime but you will need to send a real DateTime from the server

public class TestModel
{
    [JsonProperty(PropertyName = "type")]
    public int Type { get; set; }
    [JsonProperty(PropertyName = "ID")]
    public int Id {get; set;}
    [JsonProperty(PropertyName = "groupID")]
    public int GroupId {get; set;}
    [JsonProperty(PropertyName = "authorID")]
    public int AuthorId {get; set;}
    [JsonProperty(PropertyName = "contents")]
    public string Contents {get; set;}
    [JsonProperty(PropertyName = "time_stamp")]
    public string TimeStamp {get; set;}
}
ejwill
  • 141
  • 8
  • Hi, I am most likely going to implement a custom model in my code like this so thank you! However my problem still remains and, i believe, is being caused by the input being given to the deserialize function. – Matthew Swallow Dec 21 '20 at 20:51
  • @MatthewSwallow if you are able to log out the data you are getting from your server that would be helpful. You can also use an online JSON validator (you will have to use a tool to deserialize it first; maybe try doing it manually) to validate that the response is actually valid JSON. You may have to mess with your JSON settings to make it work properly – ejwill Dec 21 '20 at 21:28
  • thanks for the response, I have solved the problem now. My original post already contained the output you requested (or at the time what I thought the output was xD) – Matthew Swallow Dec 21 '20 at 22:12
  • 1
    @MatthewSwallow happy to help, and welcome to Stack Overflow. If this answer solved your issue, please mark it as accepted. – ejwill Dec 22 '20 at 01:51
0

My Issue

Thank you so much to everyone who tried to solve my issue. Feeling pretty stupid with myself, I have now solved the issue xD. Big thanks to @Jason who mentioned viewing my variables contents in Hexadecimal. I wrote a few lines to convert the variable to Hex, copied this output and threw it into an online Hex to ASCII converter to find that I was misreading the output from my program and had ignored over double the amount of data was being returned by the PHP script due to a rogue print_r() command left within the PHP from testing which outputted the array before it was JSON encoded:

Array
(
    [0] => Array
        (
            [type] => 0
            [ID] => 1
            [groupID] => 0
            [authorID] => 14
            [contents] => rfgwgfgdfssd
            [time_stamp] => 0-0-0
        )

    [1] => Array
        (
            [type] => 0
            [ID] => 2
            [groupID] => 0
            [authorID] => 14
            [contents] => whwrgthwr
            [time_stamp] => 0-0-0
        )

)
[{"type":"0","ID":"1","groupID":"0","authorID":"14","contents":"rfgwgfgdfssd","time_stamp":"0-0-0"},{"type":"0","ID":"2","groupID":"0","authorID":"14","contents":"whwrgthwr","time_stamp":"0-0-0"}]

Instead of just:

[{"type":"0","ID":"1","groupID":"0","authorID":"14","contents":"rfgwgfgdfssd","time_stamp":"0-0-0"},{"type":"0","ID":"2","groupID":"0","authorID":"14","contents":"whwrgthwr","time_stamp":"0-0-0"}]

Other Useful Possible Issues Summarised

In order to assist others in the future, I see it being only fair for me to also mention some of the things I have learned from this thread COULD have been the problem with that error.

Rogue Byte Order Marks / Zero Width Spaces

  • In C#, Unicode Characters like these can be removed with the String.Trim() function
  • Zero Width Space is always represented in unicode as U+200B
  • Byte Order Marks change based on what encoding has been used. For UTF-16, the BOM character is represented as U+FEFF
  • This means the fix for this problem is myString = myString.Trim(new char[]{'\uFEFF','\u200B'});

Incorrect Datatype / Not Sure What Datatype To Use

  • Create your own object that fits the structure you require (as shown above in @ejwill's post)
  • Deserialize into this structure

List / Dictionary Accidentally Imported From a Different Namespace

  • Make sure that none of the imported modules have a definition for List or Dictionary
  • If they do you can either:
  • Shorten your include so that only the object(s) you need from that module (IE: using System.Text; instead of using System;)
  • Or add the namespace to the front of your List / Dictionary declaration (IE: System.Collections.Generic.List<string> instead of just List<string>)
Matthew Swallow
  • 179
  • 1
  • 11