Make multiple asynchronous fetch requests and find out which request includes a string match

Question

Within a Chrome Extension i make multiple get requests and want to find out for which of the get requests the result has a match with a given string.

Because of the asynchronous call my "logging" which url i am analysing is not up to date, when i make the string match.

What i tried:

Going for r.url. But i am not sure this can be done to handover the value together with r.text(). Also i tried to make nested .then() calls, but that didnt work out.

Reproducible / example Code:

urls = ["https://stackoverflow.com/questions/tagged/javascript", "https://stackoverflow.com/questions/tagged/python"];
for (var reqNr = 0; reqNr < urls.length; reqNr++) {
    reqUrl = urls[reqNr];
    var AjaxPromise = fetch(reqUrl);
    console.log("reqUrl");
    console.log(reqUrl);    
    AjaxPromise.then(r => r.text()).then(result => {
      if(result.includes("javascript")){
        console.log(result);
        // This is the wrong Url now, because of the asynchronity
        getUrl = reqUrl;
        console.log("reqUrl2"); 
        console.log(reqUrl); // will only show the last url in loop because of the asynchronity.    

        // I could take r.url, but i am not sure i can also pass it together with result: https://stackoverflow.com/questions/28703625/how-do-you-properly-return-multiple-values-from-a-promise.
        console.log("Found match for url: ", reqUrl); // not the "correct" reqUrl
        // console.log("Found match for url: ", r.url); DOESNT WORK
      }
    }); 
  }

It might be solved dispensing the asychronity, but I would prefer to stay with the asynchronity due to Performance issues..

One way you could do it would be to declare `reqUrl` with `let` or `const`. Then you could refer to `reqUrl` even inside the `.then` statements. `let` and `const` both get scoped to a particular loop iteration, whereas `var` gets hoisted, and implicit global (which is what it appears you have here) is...well, global, so it gets overwritten every loop. — David784, Aug 22 '19 at 01:41
Possible duplicate of [JavaScript closure inside loops – simple practical example](https://stackoverflow.com/questions/750486/javascript-closure-inside-loops-simple-practical-example) — wOxxOm, Aug 22 '19 at 03:45

Ted Brownlow · Answer 1 · 2019-08-22T11:42:57.190

The most reliable way to do this is to store a unique result for each url.

async function fetchMatchesRegex(url) {
    let response = await fetch(url);
    let text = await response.text();
    let matches = text.includes("javascript");
    return {
        url:url,
        matches:matches
    }
}

function runSearch() {
    return new Promise((resolve,reject)=>{
        let urls = ["https://stackoverflow.com/questions/tagged/javascript", "https://stackoverflow.com/questions/tagged/python"];
        let promises = urls.map(url=>fetchMatchesRegex(url));
        promises.forEach(promise=>promise.then(result=>{
            if (result.matches) resolve(result.url);
        }))
    })
}

runSearch().then(url=>console.log(url));

EDIT: If you want to store all the results matching the regex, here's how

async function runMultiSearch() {
    let urls = ["https://stackoverflow.com/questions/tagged/javascript", "https://stackoverflow.com/questions/tagged/python"];
    let promises = urls.map(url=>fetchMatchesRegex(url));
    let results = await Promise.all(promises);
    return results.filter(result=>result.matches).map(result=>result.url);
}

runMultiSearch().then(urls=>console.log(urls));

First of all thank you for the answer, i upvoted as it solves the Problem with the "wrong url". Not sure if this should be a new Question, but if i want to return multiple Matches (e.g. `let matches = text.includes("Questions tagged");` should be a match for both URLs) the function Returns only the first match. If my spec was incomplete and i should open a new Question i am open to do so! — Tlatwork, Aug 22 '19 at 09:56

Make multiple asynchronous fetch requests and find out which request includes a string match

1 Answers1