Promises and Q - Completing a long series of functions with a forEach

Question

A little confused on how best to write this - hope this description is clear.

I have a forEach function in which I go through some JS object data, and for each item i execute a function:

found.forEach(function(item) {
      processData(item['userID']);
});

Within this processData function I am using a MongoDB find() call.

var processData = function(userIDSelected) {

    User.find( {_id: userIDSelected},
           {gender: 1, country:1}, function(req, foundUser) {
            processUserInfo(foundUser[0]['gender']);
    });
}

The issue I have is how do i wait for everything in forEach to complete considering that each call will be running processUserInfo in turn.

I've looked at using the Q library and Q.all however that doesnt work.

Is there a Q function that waits for everything in the long chain to complete?

Thanks

Could you not create another function that receives a list of `ObjectId` to retrieve all the users at once? — , Feb 15 '16 at 12:52

score 1 · Answer 1 · edited May 23 '17 at 11:51

Q.all:

Returns a promise that is fulfilled with an array containing the fulfillment value of each promise, or is rejected with the same rejection reason as the first promise to be rejected.

or Q.allSettled:

Returns a promise that is fulfilled with an array of promise state snapshots, but only after all the original promises have settled, i.e. become either fulfilled or rejected.

So you'd do three things:

Modify processData and possibly your call to MongoDB so that you end up with processData returning a promise for the asynchronous operation. (Apologies, I'm not familiar with MongoDB. Alnitak says that with modern versions, if you don't supply a callback, MongoDB returns a promise (nice!). Otherwise, this question and its answers may help with returning a promise for a callback-based API.)
Use map instead of forEach to get an array of the resulting promises.
Use Q.all or Q.allSettled on that array of promises.

If User.find returns a promise when no callback is specified, #1 looks something like this:

var processData = function(userIDSelected) {

    return User.find(
           {_id: userIDSelected},
           {gender: 1, country:1}
    ).then(function(req, foundUser) { // <== Check these args with the MongoDB docs!
        return processUserInfo(foundUser[0]['gender']);
    });
};

If not, you can do it yourself with Q.defer:

var processData = function(userIDSelected) {
    var d = Q.defer();
    User.find(
           {_id: userIDSelected},
           {gender: 1, country:1},
           function(req, foundUser) {
        processUserInfo(foundUser[0]['gender']);
        d.resolve(/*...data could go here...*/); // You'd use d.reject() if there were an error
    });
    return d.promise;
};

Then here's what 2 and 3 look like:

Q.all(found.map(function(item) { // Or Q.allSettled
    return processData(item);
}))
.then(...)
.catch(...);

If processData only ever uses its first argument (ignoring any extra ones), you can ditch the intermediary function:

Q.all(found.map(processData)) { // Or Q.allSettled
.then(...)
.catch(...);

...but only if processData ignores extra args, as map will pass it three (the value, its index, and the array).

the OP also needs to make the MongoDB `.find` call return a promise instead of use a callback — Alnitak, Feb 15 '16 at 12:36
Ok - could you please describe that a little more? As i just want another function to run once its find — userMod2, Feb 15 '16 at 12:44
@userMod2: I expanded things a bit based on Alnitak's comment. — T.J. Crowder, Feb 15 '16 at 12:48
@T.J.Crowder - I made a typo in my original question, processDataOptions should be processData. have updated code — userMod2, Feb 15 '16 at 12:48
@T.J.Crowder the more recent MongoDB APIs for Node will automatically return a Promise if no callback function is passed — Alnitak, Feb 15 '16 at 12:52
@Alnitak and TJ Crowder Thanks for your help, i've been trying to follow however still having issues - its not waiting the block to complete. I've attached updated code to teh bottom of the question. please could you see where I'm going wrong — userMod2, Feb 15 '16 at 17:20
Have posted the Q here: http://stackoverflow.com/questions/35415545/problems-with-promises-in-js — userMod2, Feb 15 '16 at 17:33

score 0 · Answer 2 · answered Feb 15 '16 at 17:56

The way you coded I can assume that userID is an ObjectId from MongoDB.

If that is the case, this is going to work as long as found is not empty, otherwise your users would wait for the server response forever.

processData(
  // Retrieve an object 
  // {
  //   $in: [ObjectId, ObjectId, ...]
  // }
  // 
  found.reduce(
    function(query, user) {
      return query.$in.push(user.userID), query;
    },
    {$in: []}
  )
);

You can have more info about the $in operator here.

Promises and Q - Completing a long series of functions with a forEach

2 Answers2