JSON wrapped in NULL?

Question

I'm using the API of an affiliate network (Sovrn), expecting to retrieve a product's specification using the URL.

As per their documentation, I use:

url = 'URL-goes-here'

headers = {
    "accept": "application/json",
    "authorization": "VERY-HARD-TO-GUESS"
}

response = requests.get(url, headers=headers)

The code is working, the response I get is 200, the header contains the magical content-type application/json line

when I do

print(response.text)

I get

NULL({"merchantName":"Overstock","canonicalUrl":"URL goes here","title":"product name",...});

I tested for response type of response.text, it's <class 'str'> as expected. But when I try to process the response as json:

product_details = json.load(response.text)

I get an error message:

requests.exceptions.JSONDecodeError: [Errno Expecting value]

I'm new to JSON, but I assume the error is due to the outer NULL that the (seemingly valid) data is wrapped in.

After spending a few hours searching for a solution, it seems that I must be missing something obvious, but not sure what.

Any pointers would be extremely helpful.

Is this an officially sanctioned API, or are you using someone's private API? If the latter, then yeah, it's probably wrapped in some nonsense explicitly to deter people like you. Or you've accidentally picked the *JSONP* API instead. — deceze, Jan 01 '23 at 16:36
Looking at Sovrn's documentation, you probably are indeed hitting a JSON-P API. Does your real code say `callback='NULL'` or anything like it? — user2357112, Jan 01 '23 at 16:54
@deceze: The "Try it" button on their documentation shows no JSONP wrapping with an empty "callback" field, and "NULL(...)" if I put `NULL` for `callback`. Some of the pages have `NULL` pre-populated in the `callback` field, and I think the questioner may have assumed they were actually supposed to use that value, but taking it out seems to produce a non-JSONP response. — user2357112, Jan 01 '23 at 17:02
To answer all questions, this is the official Sovrn API, and no reference is made to JSONP whatsoever. This is the Python API using JSON (the alternative solution is XML - I haven't tested that one). Thanks for all your support, @tdelaney's solution below solved the issue — Zsolt Balla, Jan 01 '23 at 19:18
There's a `callback` parameter [documented](https://i.imgur.com/SKZJzs5.png) as "JSON-P callback method name" on most of their API endpoints that produces the behavior you're seeing if you pass `'NULL'` for that parameter. — user2357112, Jan 02 '23 at 00:47

score 1 · Answer 1 · answered Jan 01 '23 at 16:38

1

You can ignore NULL( at the beginning and ); at the end by using string slicing:

product_details = json.loads(response.text[5:-2])

Additionally, you should be using json.loads() as the content is a string.

answered Jan 01 '23 at 16:38

MattDMo

100,794
21
241
231

score 1 · Accepted Answer · answered Jan 01 '23 at 16:39

1

That's clearly a bug in the API. Assuming it will be fixed after you complain, you could add a hack to your code

def sovrn_json_load_hack(json_text):
    """sovrn is returning invalid json as of (revision here)."""
    if not json_text.startswith ("NULL("):
        return json.loads(json_text)
    else:
        return json.loads(json_text[5:-2])

answered Jan 01 '23 at 16:39

tdelaney

73,364
6
83
116

This did the trick, thank you very much! I was hesitant to open a ticket with Sovrn, assuming it was my mistake, but now I will flag this to them. It's interesting no one ran into this before (or at least I couldn't trace it if someone did). Thanks, anyway, happy new year! – Zsolt Balla Jan 01 '23 at 19:12

JSON wrapped in NULL?

2 Answers2