Retrieve data from a ReadableStream object?

Question

How may I get information from a ReadableStream object?

I am using the Fetch API and I don't see this to be clear from the documentation.

The body is being returned as a ReadableStream and I would simply like to access a property within this stream. Under Response in the browser dev tools, I appear to have this information organised into properties, in the form of a JavaScript object.

fetch('http://192.168.5.6:2000/api/car', obj)
    .then((res) => {
        if(!res.ok) {
            console.log("Failure:" + res.statusText);
            throw new Error('HTTP ' + res.status);
        } else {
            console.log("Success :" + res.statusText);
            return res.body // what gives?
        }
    })

Answer 1

In order to access the data from a ReadableStream you need to call one of the conversion methods (docs available here).

As an example:

fetch('https://jsonplaceholder.typicode.com/posts/1')
  .then(function(response) {
    // The response is a Response instance.
    // You parse the data into a useable format using `.json()`
    return response.json();
  }).then(function(data) {
    // `data` is the parsed version of the JSON returned from the above endpoint.
    console.log(data);  // { "userId": 1, "id": 1, "title": "...", "body": "..." }
  });

---

EDIT: If your data return type is not JSON or you don't want JSON then use text()

As an example:

fetch('https://jsonplaceholder.typicode.com/posts/1')
  .then(function(response) {
    return response.text();
  }).then(function(data) {
    console.log(data); // this will be a string
  });

Answer 2

Some people may find an async example useful:

var response = await fetch("https://httpbin.org/ip");
var body = await response.json(); // .json() is asynchronous and therefore must be awaited

json() converts the response's body from a ReadableStream to a json object.

The await statements must be wrapped in an async function, however you can run await statements directly in the console of Chrome (as of version 62).

Answer 3

response.json() returns a Promise. Try ...

res.json().then(body => console.log(body));

where response is the result of the fetch(...)

Answer 4

Little bit late to the party but had some problems with getting something useful out from a ReadableStream produced from a Odata $batch request using the Sharepoint Framework.

Had similar issues as OP, but the solution in my case was to use a different conversion method than .json(). In my case .text() worked like a charm. Some fiddling was however necessary to get some useful JSON from the textfile.

Answer 5

For those who have a ReadableStream and want to get the text out of it, a short hack is to wrap it in a new Response (or Request) and then use the text method:

let text = await new Response(yourReadableStream).text();

Answer 6

Note that you can only read a stream once, so in some cases, you may need to clone the response in order to repeatedly read it:

fetch('example.json')
  .then(res=>res.clone().json())
  .then( json => console.log(json))

fetch('url_that_returns_text')
  .then(res=>res.clone().text())
  .then( text => console.log(text))

Answer 7

If you just want the response as text and don't want to convert it into JSON, use https://developer.mozilla.org/en-US/docs/Web/API/Body/text and then then it to get the actual result of the promise:

fetch('city-market.md')
  .then(function(response) {
    response.text().then((s) => console.log(s));
  });

or

fetch('city-market.md')
  .then(function(response) {
    return response.text();
  })
  .then(function(myText) {
    console.log(myText);
  });

Answer 8

I dislike the chaining thens. The second then does not have access to status. As stated before 'response.json()' returns a promise. Returning the then result of 'response.json()' in a acts similar to a second then. It has the added bonus of being in scope of the response.

return fetch(url, params).then(response => {
    return response.json().then(body => {
        if (response.status === 200) {
            return body
        } else {
            throw body
        }
    })
})

Answer 9

Another approach could be to consume/process the incoming stream of data in chunks:

async function toJSON(body) {
  const reader = body.getReader();
  const decoder = new TextDecoder();
  const chunks = [];

  async function read() {
    const { done, value } = await reader.read();

    if (done) {
      return JSON.parse(chunks.join(''));
    }

    const chunk = decoder.decode(value, { stream: true });
    chunks.push(chunk);
    return read();
  }

  return read();
}

const response = await fetch('https://jsonplaceholder.typicode.com/todos/1');
const jsonData = await toJSON(response.body);

console.log(jsonData);

This approach is especially useful when the incoming data is too large or if you wish to start processing the data as soon as its chunks are available.

Wrote a blog post for those interested in learning more.

Answer 10

here is how I implemented it. In this case the api is returning a ndjson as a stream, and I am reading it in chunks. In ndjson format, data is split by new lines, so each line by itself is a basic json which I parsed and added to fetchedData variable.

var fetchedData = [];

fetch('LinkGoesHere', {
    method: 'get',
    headers: {
        'Authorization': 'Bearer TokenGoesHere' // this part is irrelevant and you may not need it for your application
    }
})
.then(response => {
    if (!response.ok) {
        throw new Error(`HTTP error! Status: ${response.status}`);
    }
    return response.body.getReader();
})
.then(reader => {
    let partialData = '';

    // Read and process the NDJSON response
    return reader.read().then(function processResult(result) {
        if (result.done) {
            return;
        }

        partialData += new TextDecoder().decode(result.value, { stream: true });
        const lines = partialData.split('\n');

        for (let i = 0; i < lines.length - 1; i++) {
            const json = JSON.parse(lines[i]);
            fetchedData.push(json); // Store the parsed JSON object in the array
        }

        partialData = lines[lines.length - 1];

        return reader.read().then(processResult);
    });
})
.then(() => {
    // At this point, fetchedData contains all the parsed JSON objects
    console.log(fetchedData);
})
.catch(error => {
    console.error('Fetch error:', error);
});

Answer 11

I just had the same problem for over 12 hours before reading next, just in case this helps anyone. When using nextjs inside your _api page you will need to use JSON.stringify(whole-response) and then send it back to your page using res.send(JSON.stringify(whole-response)) and when it's received on the client side you need to translate it back into json format so that it's usable. This can be kinda figured out by reading their serialization section. Hope it helps.