Function returning before data is processes

Question

I thought for loops were blocking in Javascript, but this function is returning an empty array before the for loop finishes. Is the answer to this to setup a new function with just the for loop as a promise? If so what does that look like, the syntax for a promise is really confusing to me.

async function retrieve_s3_file(to_do_list, guid){
  var data_list = [];

  for (let i = 0; i < to_do_list.length; i++)
  {
    element = to_do_list[i];
    console.log("\n\nOutgoing request /jobs/list?guid=" + guid + "&file=" + element);
    axios.get(job_queue_url + "?guid=" + guid + "&file=" + element)
    .then(function (res){
      data_list.push(res.data);
      console.log("Inside Loop: " + JSON.stringify(data_list));
    })
    .catch(function (error){
      console.log(error);
    });
  }

  console.log("Data List: " + JSON.stringify(data_list));
  return JSON.stringify(data_list);
}

Does this answer your question? [How to return the response from an asynchronous call](https://stackoverflow.com/questions/14220321/how-to-return-the-response-from-an-asynchronous-call) which links to [How can I fetch an array of URLs with Promise.all?](https://stackoverflow.com/questions/31710768/how-can-i-fetch-an-array-of-urls-with-promise-all/67671337#67671337) — ggorlen, Apr 22 '22 at 20:54

score -1 · Accepted Answer · answered Apr 22 '22 at 20:17

Javascript is single threaded and thereby loops are usually blocking, but in this case your promises will put it's new tasks at the end of the stack. If you use await no new task should be created and it should behave as you want. But you are making it sycronus so it will be a little slower than before.

async function retrieve_s3_file(to_do_list, guid){
  var data_list = [];

  for (let i = 0; i < to_do_list.length; i++)
  {
    element = to_do_list[i];
    console.log("\n\nOutgoing request /jobs/list?guid=" + guid + "&file=" + element);
    await axios.get(job_queue_url + "?guid=" + guid + "&file=" + element)
    .then(function (res){
      data_list.push(res.data);
      console.log("Inside Loop: " + JSON.stringify(data_list));
    })
    .catch(function (error){
      console.log(error);
    });
  }

  console.log("Data List: " + JSON.stringify(data_list));
  return JSON.stringify(data_list);
}

score -1 · Answer 2 · answered Apr 22 '22 at 20:40

axios.get is a promise. In javascript a promise is non blocking, meaning the callback given in the then function will run once the promise resolve. But the promise won't block the thread.

The below example reflects the problem of the question. The console prints start end then ok. Because customPromise is a promise and it is called without the await, the caller won't wait for that promise to finish, so the caller will continue the execution and print end. Note that this promise is resolved immediately but the callback res => console.log(res) will be executed at the end because its a promise callback.

const customPromise = new Promise((resolve, reject)=>{resolve('ok')})


function nonBLockingExample (){
  console.log('start');
  customPromise.then(res => console.log(res));
  console.log('end');
}

nonBLockingExample();

Below is an example of the desired output. The caller waits for the customPromise to resolve because customPromise is called with the await keyword.

const nonBlockingPromise = new Promise((resolve, reject)=>{resolve('ok')})

async function bLockingExample (){
  console.log('start');
  await nonBlockingPromise.then(res => console.log(res));
  console.log('end');
}

bLockingExample();

So to apply the fix to your code, just await axios.get. (await axios.get(.....)

Thanks for the explanation. So just to make sure I understand, since the Axios promise is async, Javascript executes the rest of the function and then waits to get a response from the promise after the function has returned? Is that what my issue is? — aggie0642, Apr 22 '22 at 22:49

score -1 · Answer 3 · answered Apr 22 '22 at 20:40

If you use asynchronous operation it won't block.

You can use await keyword if you want Axios to wait before the request ends otherwise use promises. Collect them and using Promise.all you can get all the responses as an array when all requests are resolved.

async function retrieve_s3_file(to_do_list, guid) {
  const requests = [];
  for (const file of to_do_list) {
    const params = new URLSearchParams({ guid, file });
    const request = axios.get(`${job_queue_url}?${params.toString()}`);
    requests.push(request.then((response) => response.data));
  }

  return Promise.all(requests).then(JSON.stringify);
}

Function returning before data is processes

3 Answers3