Sequential execution of Promise.all

Question

Hi I need to execute promises one after the other how do I achieve this using promise.all any help would be awesome. Below is the sample of my code I am currently using but it executes parallel so the search will not work properly

public testData: any = (req, res) => {
    // This method is called first via API and then promise is triggerd 
    var body = req.body;

    // set up data eg 2 is repeated twice so insert 2, 5 only once into DB
    // Assuming we cant control the data and also maybe 3 maybe inside the DB
    let arrayOfData = [1,2,3,2,4,5,5];

    const promises = arrayOfData.map(this.searchAndInsert.bind(this));

    Promise.all(promises)
        .then((results) => {
            // we only get here if ALL promises fulfill
            console.log('Success', results);
            res.status(200).json({ "status": 1, "message": "Success data" });
        })
        .catch((err) => {
            // Will catch failure of first failed promise
            console.log('Failed:', err);
            res.status(200).json({ "status": 0, "message": "Failed data" });
        });
}

public searchAndInsert: any = (data) => {
    // There are database operations happening here like searching for other
    // entries in the JSON and inserting to DB
    console.log('Searching and updating', data);
    return new Promise((resolve, reject) => {
        // This is not an other function its just written her to make code readable
        if(dataExistsInDB(data) == true){
            resolve(data);
        } else {
            // This is not an other function its just written her to make code readable
            insertIntoDB(data).then() => resolve(data);
        }
    });
}

I looked up in google and saw the reduce will help I would appreciate any help on how to convert this to reduce or any method you suggest (Concurrency in .map did not work)

Have you tried `async await` that will resolve your problem. — Azeem Aslam, Feb 25 '19 at 13:29
Take a look of this link mean while I will look for problem in your existing code. https://javascript.info/async-await — Azeem Aslam, Feb 25 '19 at 13:32
Avoid the [`Promise` constructor antipattern](https://stackoverflow.com/q/23803743/1048572?What-is-the-promise-construction-antipattern-and-how-to-avoid-it) in `searchAndInsert`! — Bergi, Feb 25 '19 at 13:34
@Bergi Is that a mistake in which I wrote this code or is the ideas of promises flawed — Akshay Venugopal, Feb 25 '19 at 13:39
@AkshayVenugopal I guess your idea of promises is not complete, which led to the mistake in your code — Bergi, Feb 25 '19 at 13:43
@AkshayVenugopal you wanna go with Promiss.all approach ? Then I will share it's example. BTW Async await will keep your code tidy and clean. — Azeem Aslam, Feb 25 '19 at 13:45
I have added comments in searchAndInsert they are not other functions insert and search is written there itself just wrote a shorthand to make it more readable I still don't understand the mistake can you point it out @Bergi — Akshay Venugopal, Feb 25 '19 at 13:46
Possible duplicate of [Resolve promises one after another (i.e. in sequence)?](https://stackoverflow.com/questions/24586110/resolve-promises-one-after-another-i-e-in-sequence) — sigmus, Feb 25 '19 at 13:55

score 12 · Accepted Answer · answered Feb 25 '19 at 13:52

the Promises unfortunatelly does not allow any control of their flow. It means -> once you create new Promise, it will be doing its asynchronous parts as they like.

The Promise.all does not change it, its only purpose is that it checks all promises that you put into it and it is resolved once all of them are finished (or one of them fail).

To be able to create and control asynchronous flow, the easiest way is to wrap the creation of Promise into function and create some kind of factory method. Then instead of creating all promises upfront, you just create only one promise when you need it, wait until it is resolved and after it continue in same behaviour.

async function doAllSequentually(fnPromiseArr) {
  for (let i=0; i < fnPromiseArr.length; i++) {
    const val = await fnPromiseArr[i]();
    console.log(val);
  }
}

function createFnPromise(val) {
  return () => new Promise(resolve => resolve(val));
}

const arr = [];
for (let j=0; j < 10; j++) {
  arr.push(createFnPromise(Math.random()));
}

doAllSequentually(arr).then(() => console.log('finished'));

PS: It is also possible without async/await using standard promise-chains, but it requires to be implemented with recursion.

Using recursion will not be a problem but I am new to JS so any help would be nice — Akshay Venugopal, Feb 25 '19 at 13:59
@AkshayVenugopal - if you have support of Node 7+, just use the async/await, its easire to write and read. — libik, Feb 25 '19 at 14:01

score 3 · Answer 2 · answered Nov 18 '21 at 19:48

If anyone else cares about ESLint complaining about the use of "for" and the "no await in loop" here is a typescript ESLint friendly version of the above answer:

async function runPromisesSequentially<T>(promises: Array<Promise<T>>):Promise<Array<T>> {
  if (promises.length === 0) return [];
  const [firstElement, ...rest] = promises;
  return [await firstElement, ...(await runPromisesSequentially(rest))];
}

You can then just replace Promise.all by runPromisesSequentially.

score 2 · Answer 3 · answered Nov 23 '22 at 15:56

2

@lmX2015's answer is close but it's taking in promises that have already started executing.

A slight modification fixes it

export async function runPromisesSequentially<T>(functions: (() => Promise<T>)[]): Promise<T[]> {
  if (functions.length === 0) {
    return [];
  }
  const [first, ...rest] = functions;

  return [await first(), ...(await runPromisesSequentially(rest))];
}

answered Nov 23 '22 at 15:56

cdeutsch

3,847
27
25

Yep the other solution wouldn't work since the promises complete out of order – takanuva15 Dec 21 '22 at 22:36

score -1 · Answer 4 · answered Sep 02 '23 at 12:39

The original post asked how the poster might convert their code into a .reduce().

Here is one possible way that uses a .reduce(), processes each promise one at a time before processing the next, and adheres to most modern linting rules:

try {
  const results = promises.reduce(async (acc, curr) => {
    return [
      ...(await acc),
      await this.searchAndInsert.bind(this)(curr),
    ];
  }, []);
  // we only get here if ALL promises fulfill
  console.log('Success', results);
  res.status(200).json({ "status": 1, "message": "Success data" });
} catch (err) {
  // Will catch failure of first failed promise
  console.log('Failed:', err);
  res.status(200).json({ "status": 0, "message": "Failed data" });
}

Of course, this doesn't read extremely well, and if you have to do this one time in your code base, it is often likely you'll want to do it more than once. It's best to wrap up the reduce into a reusable function.

const mapAsyncSequential = async (listItems, mapFn) => {
    return listItems.reduce(
        async (acc, curr, i) => {
            return [
                ...acc,
                await mapFn(curr, i, list),
            ];
        },
        [],
    );
}

and then you can rewrite the first code snippet with the .reduce() line updated to use the new convenience function:

try {
  const results = await mapAsyncSeqential(promises, this.searchAndInsert.bind(this));
  // we only get here if ALL promises fulfill
  console.log('Success', results);
  res.status(200).json({ "status": 1, "message": "Success data" });
} catch (err) {
  // Will catch failure of first failed promise
  console.log('Failed:', err);
  res.status(200).json({ "status": 0, "message": "Failed data" });
}

Considering that the original Promise.all() strategy would typically work much faster, just some points to keep in mind:

Processing sequentially should be limited to situations where you absolutely need it. For example, if each promise depends on the previous one finishing, or if processing all promises at the same time causes server issues.
- If your issue is with server problems due to too many requests in sequence, another slightly more complex approach is to "chunk" your array up and process each chunk sequentially, but their contents with Promise.all().
Using a .reduce() has a memory cost. For small arrays, this is usually negligible, but if you're processing a large enough data set, you may want to write mutative code (for...of, etc) to ensure performance is not compromised.

Sequential execution of Promise.all

4 Answers4