Dynamic fetch requests in parallel

Question

I would like to know how I can do dynamic fetch requests in parallel, I have been trying to do this for 12 hours and I cant figure it out, I looked everywhere on google and StackOverflow, I really am tired.


      for (const fileindex of filelist) {
        let dcmFilename = `slice_${fileindex}.dcm`
        slices.push( {index:fileindex, filename:dcmFilename} )
          fd.append(dcmFilename, files[fileindex], dcmFilename);
        fd.append('slices', JSON.stringify(slices));
        loopcount += 1
        if (filecount == 24 || loopcount == filelist.length){
          
      if (cursorPos !== false) {
        let scaleFactor = Tegaki.scaleFactor;
        cursorPos.x = parseInt(cursorPos.x / scaleFactor);
        cursorPos.y = parseInt(cursorPos.y / scaleFactor);
        fd.append('x', cursorPos.x);
        fd.append('y', cursorPos.y)
      }

      // Get layer id from index
      if (index !== -1) {
        type = this.layerTypes[index]['id']
      }
      // switch mode from heuristic to pytorch vertebrae for vertebral bone
      if (type === 'vertebral-bone' ){
        mode='PyTorch-Vertebrae'
      }

      // Post to endpoint
      let domain = window.location.origin.replace(':8080', ':5000')
      let list = await fetch(`${domain}/segment?mode=${mode}&type=${type}`,  {    method: 'POST', body: fd   })
      let result = await list.json()
      // do something with result
      // finish then continue the loop and create new body and send a new request
     // clear formdata and continue loop
     fd =  new FormData()
}

I have the following fetch request that I send, where the body is generated for each request, It's never the same body, the body gets generated dynamically by the functions. Is there a way to send all the requests at once, then wait for them to return a response, then proceed with the rest of my code?

Show us your code that tries to send more than one request. This code just sends one request. We need to see exactly how you're sending multiple requests. — jfriend00, Jun 13 '22 at 05:03
I'm not sure how to use Promise.all() when the body is generated dynamically — Giot Dang, Jun 13 '22 at 05:07

jfriend00 · Accepted Answer · 2022-06-13T05:28:32.910

Instead of the for/of loop with await in it, you can use filelist.map() with an async callback.

Since .map() just blindly iterates the array without waiting for any returned promises, it will return an array of promises from all the async callbacks that .map() called. Those promises will initially be unfulfilled and all the fetch() operations will be "in-flight" at the same time. You can then use await Promise.all(...) on that returned array of promises to know when they are all done:

await Promise.all(filelist.map(async fileindex => {
    let dcmFilename = `slice_${fileindex}.dcm`
    slices.push({ index: fileindex, filename: dcmFilename })
    fd.append(dcmFilename, files[fileindex], dcmFilename);
    fd.append('slices', JSON.stringify(slices));
    loopcount += 1
    if (filecount == 24 || loopcount == filelist.length) {

        if (cursorPos !== false) {
            let scaleFactor = Tegaki.scaleFactor;
            cursorPos.x = parseInt(cursorPos.x / scaleFactor);
            cursorPos.y = parseInt(cursorPos.y / scaleFactor);
            fd.append('x', cursorPos.x);
            fd.append('y', cursorPos.y)
        }

        // Get layer id from index
        if (index !== -1) {
            type = this.layerTypes[index]['id']
        }
        // switch mode from heuristic to pytorch vertebrae for vertebral bone
        if (type === 'vertebral-bone') {
            mode = 'PyTorch-Vertebrae'
        }

        // Post to endpoint
        let domain = window.location.origin.replace(':8080', ':5000')

        let list = await fetch(`${domain}/segment?mode=${mode}&type=${type}`, { method: 'POST', body: fd })
        let result = await list.json()
        // do something with result
        // finish then continue the loop and create new body and send a new request
    }
}));

Note #1: Since you are now running multiple fetch operations in parallel, any code processing their results must not share variables. You have a few variables in this code that do not show their declarations. Those declarations should probably be inside this loop with let or const so there's a new variable for each iteration of the loop and you aren't sharing variables across iterations unless the variable is explicitly supposed to be accumulated across iterations. Suspicious variables in this loop with no local declaration that may be affected by parallel operation are filecount, loopcount, cursorPos, index, mode fd and type. You don't show the whole execution context here so we can't see enough to make a complete recommendation on these.

Note #2: The code that contains this if (filecount == 24 || loopcount == filelist.length) looks like it may be prone to problems. If you're trying to run some code when all the iterations are done, it would be better to run that code after the Promise.all() than trying to detect when your last iteration is done. Remember, your iterations will not necessarily proceed in order now because you're running them in parallel. They will launch in order, but not necessarily complete in order.

Dynamic fetch requests in parallel

1 Answers1