How to escape special characters programmatically while reading json in C#?

Question

In the json message, I have a field Name which can have a various combination of different special characters. Due to special characters, when I filter this property I get an error:

Unexpected character while parsing path query: !

Value of this property is not fixed. Since the combination of Special characters is not known in advance I cannot apply a specific escape sequence while applying jpath to filter a property.

For eg: if 'Name': ''!!!',then $..[?(@Name== '\'!!!')] will solve the problem. However the same field can also have values like 'Name': 'A!!!'.In this case same escape sequence will fail.

 [Test]
    public static void Test()
    {
        string json = @"{'Type': 'Contoso',
             'Products': [
                 {
                 'Name': ''!!!',
                 'Price': 99.95
                 }]
                    }";
        var jobject = JToken.Parse(json);
        string name = (string) jobject.SelectToken("$..[?(@Name== ''!!!')]");

    }

The other possible combinations of Name are 'Name': '!"!!', 'Name': ''"!!', 'Name': '$"!!' , 'Name': '/"!!' 'Name': ',!!!' etc. Is there a generic way of escaping the special characters programmatically so that I can handle all the combinations?

What the problem with 'A!!!' value? It looks like correct string — Aleks Andreev, Mar 15 '19 at 17:21
There is no problem with 'A!!!' .. I meant to say the property "Name" can begin with non special characters as well, which does not require to escape. — SKN, Mar 15 '19 at 18:02
So you only have problems with Name value, right? Please add more examples of problem input, if any. Or the only possible case is when name value looks like `''some string'`? — Aleks Andreev, Mar 15 '19 at 18:06
`'Name': ''!!!',` is invalid JSON. If the string is intended to contain a single apostrophe, it needs to be escaped. Otherwise, the duplicate apostrophe needs to be removed. — , Mar 15 '19 at 18:17
Single apostrophe in 'Name': ''!!!' is intended. But it is not escaped currently and on that I have no control on it as I'm the consumer of this json. Is there a way wherein I can escape it while reading it or any other work arounds? — SKN, Mar 15 '19 at 18:23
When present, will the unescaped single quote always be at the start of the string? Will it always be in the value of `name`? The only character causing you a problem is that unescaped single quote. None of the other special characters are cause for concern. — , Mar 15 '19 at 18:37
@SKN - You ask, *How to escape special characters programmatically while reading json in C#?*, but is your real question *How to escape special characters **in string literals inside JsonPATH queries**?* Your problem is with formatting the path passed to [`SelectToken`](https://www.newtonsoft.com/json/help/html/M_Newtonsoft_Json_Linq_JToken_SelectToken.htm), right? — dbc, Mar 15 '19 at 19:02

score 0 · Answer 1 · 2019-03-15T19:15:42.580

0

This should auto-escape an unescaped apostrophe. If the JSON contains multiple un-escaped apostrophes, it should recurse until all of them are fixed.

Really, though, you should contact whoever is generating the bad JSON and ask them to correct it.

private static JToken ParseWithAutoEscape(string json)
{
    try
    {
        return JToken.Parse(json);
    }
    catch (JsonReaderException ex)
    {
        if (!ex.Message.StartsWith("After parsing a value an unexpected character was encountered:"))
            throw;

        var lines = json.Split(new[] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries);
        var badLine = lines[ex.LineNumber - 1];

        // contains the part of the line before the apostrophe
        var before = badLine.Substring(0, ex.LinePosition - 1);

        // contains the apostrophe and what comes after.
        var after = badLine.Substring(ex.LinePosition - 1);

        lines[ex.LineNumber - 1] = $"{before}\\{after}";
        json = String.Join("\n", lines);

        // try again.
        return ParseWithAutoEscape(json);
    }
}

Usage:

var jobject = ParseWithAutoEscape(json);
var name = jobject.SelectToken("$..[?(@Name== '\\'!!!')]");
Debug.WriteLine(name);

Note that the ' needs to also be escaped in your query selector.

edited Mar 15 '19 at 19:15

answered Mar 15 '19 at 19:10

Thanks a lot @Amy for providing with the detailed solution.. I wouldn't have thought of this approach otherwise.. This solution solves part of my problem. the unsolved part is, after parsing it needs to save the value with escaped character, so that I can pass the same to the jpath without worrying the need for escaping.... – SKN Mar 18 '19 at 16:09
Aren't you querying for a name known ahead of time? – Mar 18 '19 at 16:17
Not always!... In real scenario, I would get the complete response from endpoint, then filter the name using jsonpath.. Then I will use the value thus obtained to perform some more operations... I would not know the value of name before hand... – SKN Mar 18 '19 at 16:31
Then I don't think you should be querying for a specific name, and instead should iterate over the array of products. – Mar 18 '19 at 16:34
If you found my answer helpful, please consider accepting it. Thanks. – Mar 21 '19 at 21:27

score 0 · Answer 2 · answered Mar 15 '19 at 19:11

Like Amy said, the json (string json) is the problem, not your jpath expression. It contains invalid json since 'Name': ''!!!' is not valid json. You need to escape the single quote to \'.

It needs to look like this:

    string json = @"{'Type': 'Contoso',
         'Products': [
             {
             'Name': '\'!!!',
             'Price': 99.95
             }]
                }";

If you have further doubts about json escape sequences try to have at look at this:

How to escape special characters in building a JSON string?

If what you really are talking about when you say "escaping special characters" is how to do a wildcard expression in JPath (i get this suspicion because you list a bunch of !!-ending patterns) then you might try something like this:

    string json = @"{'Type': 'Contoso',
         'Products': [
             {
             'Name': '\'!!!',
             'Price': 99.95
             }]
                }";
    var jobject = JObject.Parse(json);
    string name = (string)jobject.SelectToken("$.Products[?(@.Name =~ /.*!!$/i)].Name");

It will match a JavaScript regular expression, any name ending with !! (two exclamation marks).

How to escape special characters programmatically while reading json in C#?

2 Answers2