Capture Web Page screenshots at intervals with Puppeteer

Question

I have a Puppeteer function that plays back the loading of a site by taking screenshots at a set interval:

const getScreenshots = async (browser, url, ms, frames): Promise<string[]> => {
  const page = await browser.newPage()

  // Set screen size
  await page.setViewport({ width: 1280, height: 800 })

  await page.goto(url, {
    waitUntil: "networkidle0",
  })

  const promises = []
  return new Promise((resolve) => {
    const intervalId = setInterval(async () => {
      promises.push(
        page.screenshot({
          captureBeyondViewport: true,
          fullPage: true,
          encoding: "base64",
        })
      )
      if (promises.length >= frames) {
        clearInterval(intervalId)
        resolve(promises)
      }
    }, ms)
  })
}

const screenshotTest = async (url: string): Promise<string> => {
  const browser = await puppeteer.launch({ timeout: 100000 })
  try {
    const imgArray: any[] = await getScreenshots(browser, url, 42, 24)
    return imgArray[imgArray.length - 1]
  } finally {
    // await browser.close()
  }
}

this works well as long as the await browser.close() line remains commented out, but this allows a zombie process to continue running after the requests have completed. When I actually make this call, the function throws because the browser is being closed before the promises have resolved. This is clearly because the function is running async, though I had anticipated that it would wait for the promises to resolve before the finally block runs because the return value of the try block is awaiting them, but clearly this is not the case.

How can I rewrite this to await the promises before the browser is closed?

I believe I could do something like this:

while (true) {
  if (Promise.all(promises)) {
    await browser.close()
  }
}

but it seems really inefficient to just keep looping like that so I'm hoping I'm missing some simple way to await the promises, then close the browser.

Answer 1

Consider changing your getScreenshots() function be similar to:

const getScreenshots = async (browser, url, ms, frames) => {
  const promises = []
  const page = await browser.newPage()

  return page.goto(url).then(async function capture() {
    await new Promise(r => setTimeout(r, ms)) // delay
    promises.push(page.screenshot())
    if (promises.length < frames) {
      return capture()
    }
    const screenshots = await Promise.all(promises)
    page.close() // await should not be needed
    return screenshots
  })
}

---

Here's a similar end-to-end test:

import puppeteer from 'puppeteer'

const url = 'https://fetch-progress.anthum.com/30kbps/images/sunrise-progressive.jpg'
let browser

try {
  console.log('capturing', url)
  browser = await puppeteer.launch()
  const screenshots = await getAllScreenshots(browser, url, 500)
  console.log('captured', screenshots.length)
} finally {
  browser?.close()
}

// returns Promise<screenshot[]>
async function getAllScreenshots(browser, url, captureInterval) {
  const page = await browser.newPage()

  // Resolves when page load completes and all screenshots complete
  return new Promise(async (resolve, reject) => {
    const all = []
    let timeoutId = 1

    page.on('error', e => {
      timeoutId = clearTimeout(timeoutId)
      reject(e)
    })
    page.on('load', () => {
      timeoutId = clearTimeout(timeoutId)
      resolve(capture(page, all)) // capture final screenshot
    })

    page.goto(url)

    while (timeoutId) {
      await Promise.all([
        capture(page, all),
        new Promise(r => timeoutId = setTimeout(r, captureInterval))
      ])
    }
  }).finally(() => page.close())
}

// returns Promise<screenshot[]> after adding it to @all
async function capture(page, all) {
  all.push(await page.screenshot())
  return all
}

Answer 2

You are not awaiting the promises. You are awaiting a promise to an array which in turn contains all the screenshot promises. Use Promise.all for that:

await Promise.all(arrayOfPromises);

Or in your case:

await (await getScreenshots(browser, url, 42, 24))

And from that you can see that it did not make sense to wrap the whole thing into new Promise to begin with.

Answer 3

This seems like a case of the new Promise antipattern. Recent Node versions provide a promisified setTimeout and setInterval that lets you avoid callbacks.

For example, with setTimeout:

const {setTimeout} = require("node:timers/promises");

const getScreenshots = async (
  browser,
  url,
  ms,
  frames
): Promise<string[]> => {
  const page = await browser.newPage();
  await page.setViewport({width: 1280, height: 800});
  await page.goto(url, {waitUntil: "networkidle0"});

  const screenshots = [];

  for (let i = 0; i < frames; i++) {
    const screenshot = await page.screenshot({
      captureBeyondViewport: true,
      fullPage: true,
      encoding: "base64",
    });
    screenshots.push(screenshot);
    await setTimeout(ms);
  }

  return screenshots;
};

With setInterval:

const {setInterval} = require("node:timers/promises");

const getScreenshots = async (
  browser,
  url,
  ms,
  frames
): Promise<string[]> => {
  const page = await browser.newPage();
  await page.setViewport({width: 1280, height: 800});
  await page.goto(url, {waitUntil: "networkidle0"});

  const screenshots = [];

  for await (const startTime of setInterval(ms)) {
    const screenshot = await page.screenshot({
      captureBeyondViewport: true,
      fullPage: true,
      encoding: "base64",
    });
    screenshots.push(screenshot);

    if (screenshots.length >= frames) {
      return screenshots;
    }
  }
};

The calling code is the same, with browser.close() uncommented.

Note that any solution with setTimeout or setInterval will drift over time. Taking screenshots is a complex, non-instantaneous subprocess call anyway, so I imagine it'll be diminishing returns to attempt this, but you could try a tight requestAnimationFrame loop with a performance.now() call and drift correction.

In addition to new Promise, other red flags are using async on a function that doesn't have an await in it, using async on a new Promise and making a setInterval callback async. Even if you don't have a newer Node version or don't have utils.promisify (such as in the browser), it's better to bury the promisification in a one-off function and keep your mainline code callback-free:

const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));

Other minor suggestions:

• It's a bit strange that you create an array of screenshots, then throw them all away except for the last one (you can more idiomatically access the last element of an array with .at(-1)).
• When you have more than a few arguments, as in getScreenshots(browser, url, 42, 24), the recommended approach is to switch to a configuration object, like getScreenshots(browser, url, {ms: 42, frames: 24}) to keep the code readable.
• I generally prefer my Puppeteer helper functions to accept a page rather than a whole browser. This allows for maximum reusability because the callee isn't forced to create a new page. The caller can set whichever settings and URL on the page ahead of the screenshot call rather than passing them in as parameters.

Here's a complete, runnable example with the above suggestions applied:

const fs = require("node:fs/promises");
const puppeteer = require("puppeteer");
const {setInterval} = require("timers/promises");

const getScreenshots = async (page, opts = {ms: 1000, frames: 10}) => {
  const screenshots = [];

  for await (const startTime of setInterval(opts.ms)) {
    const screenshot = await page.screenshot({
      captureBeyondViewport: true,
      fullPage: true,
      encoding: "base64",
    });
    screenshots.push(screenshot);

    if (screenshots.length >= opts.frames) {
      return screenshots;
    }
  }
};

// Driver code for testing:
const html = `<!DOCTYPE html>
<html>
<body>
<h1></h1>
<script>
let i = 0;
setInterval(() => {
  document.querySelector("h1").textContent = ++i;
}, 10);
</script>
</body>
</html>
`;

let browser;
(async () => {
  browser = await puppeteer.launch();
  const [page] = await browser.pages();
  await page.setContent(html);
  const screenshots = await getScreenshots(page, {ms: 100, frames: 10});
  console.log(screenshots.length); // => 10
  const gallery = `<!DOCTYPE html><html><body>
  ${screenshots.map(e => `
    <img alt="test screenshot" src="data:image/png;base64,${e}">
  `)}
  </body></html>`;
  await fs.writeFile("test.html", gallery);
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close());

Open test.html in your browser to see 10 screenshots at different intervals.

Note how removing newPage from the function gives the caller the ability to shoot screenshots on a setContent rather than a goto.