-1

I am attempting to scrape news articles from a site using fetch and promise all. Example found here

Here is the code I am using.

        var openArticles = function(urlArray){

              Promise.all(urlArray.map(u=>fetch(u))).then(responses =>
                Promise.all(responses.map(res => res.text()))
                ).then(function(html){
                  console.log(html[0])
                 
              })
         };

I am not too familiar with arrow function expressions and am having a difficult time with adding error handling for 500 and timeout errors. I tried putting code in try catch block but that did not help, and I am not sure where or how to enter if statement to check response status. Is there a way to rewrite this using standard function name() format?

Roamer-1888
  • 19,138
  • 5
  • 33
  • 44
user1740058
  • 45
  • 1
  • 9
  • Does this answer your question? [How do I handle errors with promises?](https://stackoverflow.com/questions/21800010/how-do-i-handle-errors-with-promises) – Rojo Oct 02 '21 at 19:19
  • To be honest - No - not very helpful. – user1740058 Oct 02 '21 at 19:33
  • Just add `.catch(() => {/* code here */});` after the `.then()`... – Rojo Oct 02 '21 at 19:36
  • This was the first thing I tried. I tested with both then statements, as well as after the fetch. Neither of these caught the error. – user1740058 Oct 02 '21 at 20:35
  • Sorry; [`fetch` (MDN)](https://developer.mozilla.org/en-US/docs/Web/API/fetch) - _"A `fetch()` promise does not reject on HTTP errors (`404`, etc.). Instead, a `then() handler must check the `Response.ok` and/or `Response.status` properties."_ – Rojo Oct 02 '21 at 20:40
  • Yes, I have used that method before, the problem is "where" to insert it in a function with this style syntax. – user1740058 Oct 02 '21 at 22:34
  • See the [first example](https://developer.mozilla.org/en-US/docs/Web/API/fetch#examples) and there is also [this question](https://stackoverflow.com/questions/50330795/fetch-api-error-handling) – Rojo Oct 02 '21 at 23:37

1 Answers1

0

Ok, here is solution for now. Sorry it's ugly syntax, I'm sure there's a better way to write this.

Promise.all(urls.map(u=>fetch(u).then((response) =>{
   if(response.status != 200){
     console.log(response.status);
     // do something
    return;
  }
  else{
   return response;
  }
 }))).then(responses => Promise.all(responses.map(res => res.text())) 
).then(function(html){
 //do something else
})

Basically I had been adding the "then" statement in the wrong position. Thanks for the help

user1740058
  • 45
  • 1
  • 9
  • Instead of checking for the status code, you should check either `response.status >= 200 && response.status < 300` or just [`response.ok`](https://developer.mozilla.org/en-US/docs/Web/API/Response/ok). Every status code between 200-299 (inclusive) is a successful status. IMO, a few of the 300s are also "successful". You can read more about [status codes on MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status). – Rojo Oct 02 '21 at 23:43
  • Thanks for the advice. I actually need to check for 500 error in order to create custom response object to return. – user1740058 Oct 03 '21 at 18:44