JSON not parsing fetch response data correctly

Question

I am fetching data from an API that responds with a JSON object containing an array of objects containing another array. Something like

data: {
   items: [
      0: {
         holdings: [
            0: {...values...}
         ]
      }
   ]
}

The "items" array contains several objects but 1 of these objects keeps getting parsed incorrectly. The holdings of the first item in the array is only 24 items long, but it should be 31. Every other JSON object returns correctly with the right length of the holdings array except this one.

It returns correctly from:

The API docs website
Postman response
Chrome network tab
Chrome console tab*

It only returns correctly in a console.log() if I use response.text() in the fetch. So I know I'm getting the correct data from the API itself. The error comes from after it's been parsed into a JSON object. Furthermore, it sometimes gets parsed correctly, but once I pass that JSON object to another function, it goes from 31 to 24 again, without fail.

Here's my fetch:

return await fetch(
          <URL>,
            {
              method: 'GET',
              headers: {
                'Content-Type': 'application/json'
              }
            }
        )
          .then((res) => {
            return res.json(); //changing this to res.text() returns correct data, but unusable
          })
          .then((resData) => {
            if(!resData.error) {
              return resData;
            }
          });

EDIT:

Here's the correct response, this time from Postman. I trimmed it down to fit in the post:

{
    "data": {
        "address": "0x22...",
        "updated_at": "2022-10-15T18:48:07.076779009Z",
        "next_update_at": "2022-10-15T18:53:07.076779139Z",
        "quote_currency": "USD",
        "chain_id": 1,
        "items": [
            {
                "contract_decimals": 6,
                "contract_name": "USD Coin",
                "contract_ticker_symbol": "USDC",
                "contract_address": "0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48",
                "supports_erc": null,
                "logo_url": "https://logos.covalenthq.com/tokens/1/0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48.png",
                "holdings": [
                    {
                        "timestamp": "2022-10-15T00:00:00Z",
                        "quote_rate": 1.000501,
                        "open": {
                            "balance": "127096912",
                            "quote": 127.16059
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 127.16059
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 127.16059
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 127.16059
                        }
                    },
                    {
                        "timestamp": "2022-10-14T00:00:00Z",
                        "quote_rate": 1.0112872,
                        "open": {
                            "balance": "127096912",
                            "quote": 128.53148
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 128.53148
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 128.53148
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 128.53148
                        }
                    },
                    {
                        "timestamp": "2022-10-13T00:00:00Z",
                        "quote_rate": 0.98853105,
                        "open": {
                            "balance": "127096912",
                            "quote": 125.639244
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 125.639244
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 125.639244
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 125.639244
                        }
                    },
                    {
                        "timestamp": "2022-10-12T00:00:00Z",
                        "quote_rate": 0.99525076,
                        "open": {
                            "balance": "127096912",
                            "quote": 126.49329
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 126.49329
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 126.49329
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 126.49329
                        }
                    },
                    {
                        "timestamp": "2022-10-11T00:00:00Z",
                        "quote_rate": 1.0054721,
                        "open": {
                            "balance": "127096912",
                            "quote": 127.79239
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 127.79239
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 127.79239
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 127.79239
                        }
                    },
                    {
                        "timestamp": "2022-10-10T00:00:00Z",
                        "quote_rate": 1.0150021,
                        "open": {
                            "balance": "127096912",
                            "quote": 129.00363
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 129.00363
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 129.00363
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 129.00363
                        }
                    },
                    {
                        "timestamp": "2022-10-09T00:00:00Z",
                        "quote_rate": 0.99896634,
                        "open": {
                            "balance": "127096912",
                            "quote": 126.96553
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 126.96553
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 126.96553
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 126.96553
                        }
                    },
                    {
                        "timestamp": "2022-10-08T00:00:00Z",
                        "quote_rate": 1.0020251,
                        "open": {
                            "balance": "127096912",
                            "quote": 127.354294
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 127.354294
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 127.354294
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 127.354294
                        }
                    },
                    {
                        "timestamp": "2022-10-07T00:00:00Z",
                        "quote_rate": 1.0274968,
                        "open": {
                            "balance": "127096912",
                            "quote": 130.59167
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 130.59167
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 130.59167
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 130.59167
                        }
                    },
                    {
                        "timestamp": "2022-10-06T00:00:00Z",
                        "quote_rate": 1.0117313,
                        "open": {
                            "balance": "127096912",
                            "quote": 128.58792
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 128.58792
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 128.58792
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 128.58792
                        }
                    },
                    {
                        "timestamp": "2022-10-05T00:00:00Z",
                        "quote_rate": 1.009318,
                        "open": {
                            "balance": "127096912",
                            "quote": 128.2812
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 128.2812
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 128.2812
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 128.2812
                        }
                    },
                    {
                        "timestamp": "2022-10-04T00:00:00Z",
                        "quote_rate": 0.9641742,
                        "open": {
                            "balance": "127096912",
                            "quote": 122.543564
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 122.543564
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 122.543564
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 122.543564
                        }
                    },
                    {
                        "timestamp": "2022-10-03T00:00:00Z",
                        "quote_rate": 0.9716864,
                        "open": {
                            "balance": "127096912",
                            "quote": 123.49834
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 123.49834
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 123.49834
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 123.49834
                        }
                    },
                    {
                        "timestamp": "2022-10-02T00:00:00Z",
                        "quote_rate": 1.0129485,
                        "open": {
                            "balance": "127096912",
                            "quote": 128.74263
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 128.74263
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 128.74263
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 128.74263
                        }
                    },
                    {
                        "timestamp": "2022-10-01T00:00:00Z",
                        "quote_rate": 1.0076605,
                        "open": {
                            "balance": "127096912",
                            "quote": 128.07054
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 128.07054
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 128.07054
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 128.07054
                        }
                    },
                    {
                        "timestamp": "2022-09-30T00:00:00Z",
                        "quote_rate": 1.0058662,
                        "open": {
                            "balance": "127096912",
                            "quote": 127.84248
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 127.84248
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 127.84248
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 127.84248
                        }
                    },
                    {
                        "timestamp": "2022-09-29T00:00:00Z",
                        "quote_rate": 0.99356514,
                        "open": {
                            "balance": "127096912",
                            "quote": 126.27906
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 126.27906
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 126.27906
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 126.27906
                        }
                    },
                    {
                        "timestamp": "2022-09-28T00:00:00Z",
                        "quote_rate": 0.9847916,
                        "open": {
                            "balance": "127096912",
                            "quote": 125.16396
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 125.16396
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 125.16396
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 125.16396
                        }
                    },
                    {
                        "timestamp": "2022-09-27T00:00:00Z",
                        "quote_rate": 1.005657,
                        "open": {
                            "balance": "127096912",
                            "quote": 127.81589
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 127.81589
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 127.81589
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 127.81589
                        }
                    },
                    {
                        "timestamp": "2022-09-26T00:00:00Z",
                        "quote_rate": 0.9778665,
                        "open": {
                            "balance": "127096912",
                            "quote": 124.283806
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 124.283806
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 124.283806
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 124.283806
                        }
                    },
                    {
                        "timestamp": "2022-09-25T00:00:00Z",
                        "quote_rate": 1.0082643,
                        "open": {
                            "balance": "127096912",
                            "quote": 128.14728
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 128.14728
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 128.14728
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 128.14728
                        }
                    },
                    {
                        "timestamp": "2022-09-24T00:00:00Z",
                        "quote_rate": 1.0187384,
                        "open": {
                            "balance": "127096912",
                            "quote": 129.4785
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 129.4785
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 129.4785
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 129.4785
                        }
                    },
                    {
                        "timestamp": "2022-09-23T00:00:00Z",
                        "quote_rate": 1.0089662,
                        "open": {
                            "balance": "127096912",
                            "quote": 128.23648
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 128.23648
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 128.23648
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 128.23648
                        }
                    },
                    {
                        "timestamp": "2022-09-22T00:00:00Z",
                        "quote_rate": 0.9554906,
                        "open": {
                            "balance": "127096912",
                            "quote": 121.4399
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 121.4399
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 121.4399
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 121.4399
                        }
                    },
                    {
                        "timestamp": "2022-09-21T00:00:00Z",
                        "quote_rate": 1.0179983,
                        "open": {
                            "balance": "127096912",
                            "quote": 129.38445
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 129.38445
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 129.38445
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 129.38445
                        }
                    },
                    {
                        "timestamp": "2022-09-20T00:00:00Z",
                        "quote_rate": 1.0370787,
                        "open": {
                            "balance": "127096912",
                            "quote": 131.8095
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 131.8095
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 131.8095
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 131.8095
                        }
                    },
                    {
                        "timestamp": "2022-09-19T00:00:00Z",
                        "quote_rate": 0.9934928,
                        "open": {
                            "balance": "127096912",
                            "quote": 126.26986
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 126.26986
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 126.26986
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 126.26986
                        }
                    },
                    {
                        "timestamp": "2022-09-18T00:00:00Z",
                        "quote_rate": 1.0368719,
                        "open": {
                            "balance": "127096912",
                            "quote": 131.78322
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 131.78322
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 131.78322
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 131.78322
                        }
                    },
                    {
                        "timestamp": "2022-09-17T00:00:00Z",
                        "quote_rate": 0.9822457,
                        "open": {
                            "balance": "127096912",
                            "quote": 124.84039
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 124.84039
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 124.84039
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 124.84039
                        }
                    },
                    {
                        "timestamp": "2022-09-16T00:00:00Z",
                        "quote_rate": 0.9967559,
                        "open": {
                            "balance": "127096912",
                            "quote": 126.68459
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 126.68459
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 126.68459
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 126.68459
                        }
                    },
                    {
                        "timestamp": "2022-09-15T00:00:00Z",
                        "quote_rate": 1.0280845,
                        "open": {
                            "balance": "127096912",
                            "quote": 130.66637
                        },
                        "high": {
                            "balance": "127096912",
                            "quote": 130.66637
                        },
                        "low": {
                            "balance": "127096912",
                            "quote": 130.66637
                        },
                        "close": {
                            "balance": "127096912",
                            "quote": 130.66637
                        }
                    }
                ]
            },
        ],
        "pagination": null
    },
    "error": false,
    "error_message": null,
    "error_code": null
}

It sounds like there is something inherently wrong about the response from your endpoint: are you sure it is valid JSON? What happens if you pass the response to a JSON validator? — Terry, Oct 15 '22 at 20:39
Please attach the exact string that comes back from the server and which you are trying to convert — Haim Abeles, Oct 15 '22 at 20:44
By guess is that you receive questionable JSON with duplicate keys, something like `{"a": 1, "b": 2, "c": 3, "a": 4}`. If you put this example string into `JSON.parse` you will get `{a: 4, b: 2, c: 3}`, because the original JSON string had a duplicate key `a` in the object. [This is not forbidden per se](https://stackoverflow.com/questions/21832701/does-json-syntax-allow-duplicate-keys-in-an-object) and the JSON is still valid with duplicate keys, but many languages - including JS - won't be able to make more than one of the values accessible then, which is why it's not recommended. — CherryDT, Oct 15 '22 at 20:48
Another thing to rule out is that JSON does not encode named properties of Array objects that are not subscripts of the the array. E.G. `[1,2,3].id = "idValue"` is valid JavaScript but the `id` property is ignored when serializing the array into JSON.. — traktor, Oct 15 '22 at 23:16
I added the correct response from the API @coderpolo. Basically the top 7 elements in the 'holdings' array get removed, all from 10-08 on. Not sure why, I get dozens more and none of them have this issue, only this one — Josh Kroslowitz, Oct 16 '22 at 11:10
when i try to json decode online your json, it says its not valid so i still wonder why, i didn't find the issue yet @JoshKroslowitz — coderpolo, Oct 16 '22 at 19:18

score 0 · Answer 1 · answered Oct 16 '22 at 19:24

0

I found out that your data.items collection containing only one item has a comma after the closing bracket here :

      }, <- this little fellow
  ],
  "pagination": null

if you remove it, the json seem valid. I think it has to do with the trailing comma

i debuged it here : https://jsonparser.org/

answered Oct 16 '22 at 19:24

coderpolo

307
3
12

Thanks for trying but if I take the data from Postman and put it in jsonparser.org it parses correctly already, without changing anything. As I said, sometimes I do get the correct parsed json but if I pass it to another function it changes. Sometimes it's parsed incorrectly in the fetch. I really don't know what's going on – Josh Kroslowitz Oct 16 '22 at 19:42
dammit its really weird, are you really sure the response json data is consistent ? – coderpolo Oct 16 '22 at 19:44
Yes, it's just this one item, all the others parse just fine. Like I said in the OP, if I use response.text() it's correct every time, then when I parse it it's also fine sometimes, but if I pass it to another function it's missing 7 items in the holdings array from that first item. – Josh Kroslowitz Oct 17 '22 at 07:09

JSON not parsing fetch response data correctly

1 Answers1