How can I run promises in sequence and dynamically

Question

In my web application I can run blocks of code (it creates a promise and waits for the result to come out). Every time a user runs a paragraph, I add it's id to an array and run it sequentially.

runSequentially(paragraphsId) {
    paragraphsId.reduce((promise, paragraphId) => {
        return promise.then(() => this.runParagraph(paragraphId))
    }, Promise.resolve())
}

addToQueue(paragraphId) {
    if (this.state.runQueue.indexOf(paragraphId) === -1) {
        this.setState({
            runQueue: [...this.state.runQueue, paragraphId]
        }, () => this.runSequentially(this.state.runQueue))
    }
}

runParagraph(paragraphId) {
    const newParagraphResults = { ...this.state.paragraphResults }
    delete newParagraphResults[paragraphId]

    const newParagraphs = { ...this.state.paragraphs }
    const newParagraph = newParagraphs[paragraphId]
    newParagraph.isRunning = true
    newParagraph.status = 'running'
    this.setState({
        paragraphs: newParagraphs,
        paragraphResults: newParagraphResults
    })

    const paragraphs = [
        {
            identifiers: { id: paragraphId },
            title: newParagraph.title,
            source: newParagraph.source
        }
    ]

    const notebookLibraries = Object.values(this.state.notebookLibraries)

    this.runController = new AbortController()
    return this.service.notebookRun(this.notebookId, paragraphs, notebookLibraries, this.runController)
        .then(result => {
            Object.entries(result.paragraphs).forEach(entry => {
                if (entry[0] === 'default_paragraph') {
                    return
                }
                const paragraphId = entry[0]
                const paragraphResult = entry[1]
                newParagraphResults[paragraphId] = paragraphResult
                paragraphResult.exception ? this.setParagraph(paragraphId, { status: 'failed' }) :
                    this.setParagraph(paragraphId, { status: 'passed' })
            })
            this.setState({ paragraphResults: newParagraphResults })
        })
        .catch((error) => {
            if (error.name === 'AbortError') {
                return Promise.reject(error)
            }
            const message = `Execution failed for reason: ${error.reason}.`
            this.handleServiceError('notebook', 'run', message)
        })
        .finally(() => {
            const newRunQueue = [ ...this.state.runQueue ]
            newRunQueue.shift()
            this.setParagraph(paragraphId, { isRunning: false })
            this.setState({ runQueue: newRunQueue })
        })
}

When a user runs a paragraph we call addToQueue which then calls runSequentially. We shift the queue when a promise is settled (in the runParagraph method) but if we run another paragraph before the first one has finished this will iterate over the same promise twice.

How would you handle this dynamic queue of promises ? Could recursivity work in this case ?

How about async/await? https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function — sametcodes, Dec 28 '18 at 14:14
Possible duplicate of [How to execute promises sequentially, passing the parameters from an array?](https://stackoverflow.com/questions/43082934/how-to-execute-promises-sequentially-passing-the-parameters-from-an-array) — lukaleli, Dec 28 '18 at 14:16
@lukaleli I don't think it's a strict duplicate, since here the OP seems to want to be able to dynamically add to the queue and the question you linked is for a static array. — Jared Smith, Dec 28 '18 at 14:19
Can you please show the runParagraph() code as well? You could like, as I think you mean with recursively, not chain all the runParagraph promises together with a reduction and have the resolution of the previous promise, start running the next, so that you have time to remove a block from a queue. There's no reason to already make the promises if they have to wait for eachothers resolution. — Shilly, Dec 28 '18 at 14:25
Consider: A promise chain is effectively a queue. By using a promise chain as your queue, instead of Array, then adding becomes trivial, "running" becomes automatic and the need for status flags should disappear. Initialise the queue with `Promise.resolve()`, then append `.then(successFn).catch(errorFn)` to cause `successFn` to run when the chain advances to that point. Unless you want errors to terminate processing, then ensure they are caught (and not re-thrown); if necessary, send some suitable default object down the success path to inform the next step in the chain. — Roamer-1888, Dec 28 '18 at 18:05

score 2 · Accepted Answer · answered Dec 28 '18 at 14:28

You should initialize another property (perhaps queue is not the best name since you already have state.runQueue) in your class to Promise.resolve(), and let that be the pending promise of your sequential queue. Then you can do something like this:

runSequentially(...paragraphsId) {
  this.queue = paragraphsId.reduce((promise, paragraphId) => {
    return promise.then(() => this.runParagraph(paragraphId))
  }, this.queue)
}

addToQueue(paragraphId) {
  if (this.state.runQueue.indexOf(paragraphId) === -1) {
    this.setState({
      runQueue: [...this.state.runQueue, paragraphId]
    }, () => this.runSequentially(paragraphId))
  }
}

runSequentially() now accepts incremental updates rather than the entire runQueue, and you shouldn't store queue on your state variable because the promise itself doesn't affect your render, so it's safe.

score 2 · Answer 2 · answered Dec 28 '18 at 14:32

if we run another paragraph before the first one has finished this will iterate over the same promise twice.

You will need to keep the promise queue as a property in your state, instead of creating a new Promise.resolve() every time you call runSequentially. See here for an example implementation.

If you want to manage your queue strictly through setState, you should not need a runSequentially method at all. runParagraph itself would a) check whether it already is running and b) when finished dequeue the next element from the array and call runParagraph again until there a none left.

How can I run promises in sequence and dynamically

2 Answers2