Awaiting a function that calls an async function recursively

Question

I have a function that looks like this:

function populateMap(directory: string, map, StringMap) {
    fs.promises.readdir(directory).then(files: string[]) => {
        files.forEach(file: string) => {
            const fullPath = path.join(directory, file);
            fs.stat(fullPath, (err: any, stats: any) => {
                if (stats.isDirectory()) {
                   populateFileMap(fullPath, fileMap);
                } else {
                   fileMap[file] = fullPath;
                }
            });
        });
    });
}

What I want to do is recursively walk through the parent directory and store a map of file names to their paths. I know this is working because if I put a console.log(fileMap) under fileMap[file] = fullPath, after the deepest file in the directory, the list is properly populated.

In the file that calls this function, I want to be able to have the full map as such

function populateMapWrapper(dir: string) {
    const fileMap: StringMap = {};

    populateMap(dir, fileMap);

    //fileMap should be correctly populated here
}

I've tried making populateMap asynchronous, adding a .then() to where it's called in the wrapper function, but if I console.log(fileMap) in the then() function, the fileMap is empty.

I'm not sure if this is because of how javascript passes variables or if there's a gap in my understanding of promises, but I'm wondering if there's an alternative way to do this.

Yes, converting the callback style to promises is a good idea, as is making it an `async function` where you can `await` those promises. And [don't use `forEach`](https://stackoverflow.com/q/37576685/1048572). — Bergi, May 21 '20 at 11:39

ccarton · Accepted Answer · 2020-08-25T10:47:53.353

One problem is that fs.stat doesn't return a promise. You need to also use fs.promises.stat. Also, when working with promises be careful about using forEach, because it doesn't await for each of the forEach callbacks. You could instead use map with Promise.all()

One solution:

function populateMap(directory: string, map) {
  return fs.promises.readdir(directory).then((files: string[]) => {
    return Promise.all(
      files.map((file: string) => {
        const fullPath = path.join(directory, file);
        return fs.promises.stat(fullPath).then(stats => {
          if (stats.isDirectory()) {
            return populateMap(fullPath, map);
          } else {
            map[file] = fullPath;
          }
        })
      }))
  })
}

Then you would have to use await in the wrapper:

async function populateMapWrapper(dir: string) {
    const fileMap: StringMap = {};

    await populateMap(dir, fileMap);

    //fileMap should be correctly populated here
}

However, a more readable solution would be to make use of await whenever possible. Something like:

async function populateMap (directory: string, map) {
  const files = await fs.promises.readdir(directory)
  for (const file of files) {
    const fullPath = path.join(directory, file)
    const stats = await fs.promises.stat(fullPath)
    if (stats.isDirectory()) {
      await populateMap(fullPath, map)
    } else {
      map[file] = fullPath
    }
  }
}

Thanks for the detailed answer. I was able to follow your suggestions and it worked fine. — T95, May 21 '20 at 12:28

Awaiting a function that calls an async function recursively

1 Answers1