Async.forEach runs in parallel but it is losing the sort order in the original array

Question

I am trying to iterate over a collection in parallel and I was using async.forEach() for it, but I need to maintain the sort order too.

async.forEach(books, function (book, bookCallback) {
        findBook(account, userinfo, permission, function (error, foundBook) {
            if (error) {
                bookCallback(error);
            } else if (foundBook) {
                bookCallback(account);
                bookCallback(null);
            } else {
                bookCallback(null);
            }
        });
    }, function (err) {
        if (err) {
            callback(err, null);
        } else {
            callback(null, finaCallback);
        }
    });

I have tried with async.map, async.groupBy, but even they did not maintain sort order. Only when I tried async.eachSeries order was maintained but it was not running in parallel.

Why do it in parallel if you need control over execution order? — darksmurf, Mar 06 '19 at 06:07
By sort order you mean execution order ? If yes, running in parallel won't be the solution then. — Farhan Tahir, Mar 06 '19 at 06:11
@darksmurf The collection is huge and internally its another API call so I need to run that in parallel — Lucifer007, Mar 06 '19 at 06:21
@FarhanTahir Yes I am not worried about execution order. But I am worried about order is not maintained in books array. Before this method call this array had say ['Book1','Book2','Book3'] order, but after this call ends my books array is['Book1','Book3','Book2']. I want sorted in array to be maintained. — Lucifer007, Mar 06 '19 at 06:25
Please be more explicit by what you mean by _sort order_. Are you saying that you need to process the result callback in order? Or that you need to execute the functions in order? By running the functions in parallel, there is no guarantee of any ordering whatsoever. — Andrew Eisenberg, Mar 06 '19 at 06:25
@AndrewEisenberg So as per your comments my understanding is , it is impossible to maintain the sort order in books array if I want to iterate over the collection in parallel. I should stick to async.eachSeries? — Lucifer007, Mar 06 '19 at 06:28
Not exactly. If the ordering of the result array is all you need, then you can use standard async functions and native Promises to achieve this. Writing up an answer now. — Andrew Eisenberg, Mar 06 '19 at 06:33

score 1 · Answer 1 · answered Mar 06 '19 at 06:34

You actually want async.parallel in this case, but you basically need to transform your input to that first:

let tasks = [ ...accounts.entries() ].map(([k, account]) =>
  ({ 
    [k]: (callback) => findbook(account, userinfo, permission, callback)
  })
);

async.parallel(tasks, (err,results) => {
    if (err) throw err;          // error handling of some sort
    // reconstruct array order by index position
    results = accounts.map((e,i) => results[i])
})

Alternately, you can wrap your method that accepts a callback with a Promise and use Promise.all, which is parallel and respects the same order in results as the input used:

Promise.all(
  accounts.map(account => new Promise((resolve,reject) => 
    findbook(account, userinfo, permission, (err, result) => {
      if (err) reject(err);
      resolve(result)
    })
  ))
).then(results => {
  // results have original order
}).catch(err => console.error(err))

score 0 · Answer 2 · answered Mar 06 '19 at 06:34

I am not sure which library async is coming from, but assuming you are using a modern version of node, you don't need a special library to handle async calls. Here's what you can do:

async function getBooks() {
  let result = await Promise.all(books.map(book => {
    return new Promise((resolve, reject) => {
      findBook(account, userinfo, permission, function (error, foundBook) {
        if (error) {
          reject(error);
        } else if (foundBook) {
          resolve(account);
        } else {
          resolve(null);
        }
      });  
    });
  return result;
}

And result will be an array of results in the same order of the original books array.

The getBooks function returns a promise that is either rejected or resolved. If I understand your problem correctly, this should solve it.

@ AndrewEisenberg I will surely try this now, but before that I just wondering that iteration over books collection is in parallel right? — Lucifer007, Mar 06 '19 at 06:41

Async.forEach runs in parallel but it is losing the sort order in the original array

2 Answers2