How await a recursive Promise in Javascript

Question

I have written a recursive Promise in javascript which seems to be working fine but I wanted to test it using setTimeout() to be sure that I'm awaiting correctly before continuing with the execution. Here is the gist of my code:

try{
  await renameFiles(); // <-- await here
  console.log("do other stuff");
}
catch(){
}

const renameFiles = (path) => {
  return new Promise(resolve => {
    console.log("Renaming files...");

    fs.readdirSync(path).forEach(file) => {
      // if file is a directory ...
      let newPath = path.join(path, file);
      resolve( renameFiles(newPath) ); // <- recursion here!
      // else rename file ...
    }
    resolve();
  })

I've tested it with setTimeout() like this:

const renameFiles = () => {
  return new Promise(resolve => {
    setTimeout(() => {
    // all previous code goes here
    },2000)
  }
}

and the output is:

"Renaming files..."
"Renaming files..."
// bunch of renaming files...
"do other stuff"
"Renaming files..."
"Renaming files..."

So it looks like it's awaiting for a bit but then it continues the execution at some point.

I'm also doubting I'm testing it wrong. Any idea where the problem may be?

Answer 1

As already mentioned - multiple resolve invocations don't make sense. However that is not the only problem in the code. Root invocation got resolved when its recursive call started for first sub directory. This code will process directories in hierarchical order

rename.js

const fs = require('fs');
const path = require('path');

const inputPath = path.resolve(process.argv[2]);
const newName = 'bar.txt';

async function renameFiles(filePath) {
    for (const file of fs.readdirSync(filePath)) {
        const newPath = path.join(filePath, file);
        const descriptor = fs.lstatSync(newPath);
        if (descriptor.isDirectory()) {
            await renameFiles(newPath)
        } else if (descriptor.isFile()) {
            await renameFile(file);
        }
    }
}

async function renameFile(file) {
    console.log(`Renaming ${file} to ${newName}`)
    return new Promise(resolve => {
       setTimeout(() => {
           console.log(`Renamed ${file} to ${newName}`)
           resolve();
       }, 300)
    });
}

async function main() {
    console.log(`Renaming all files in ${inputPath} to ${newName}`);
    await renameFiles(inputPath);
    console.log('Finished');
}

main();

you can run it like

node rename.js relativeFolderName

or if order doesn't matter, then you can use map and Promise.all as mentioned by @Tiago Coelho

const renameFiles = async path => {
    const renamePromises = fs.readdirSync(path).map(file => {
      if (isDirectory(file)) {
          const newPath = path.join(path, file);
          return renameFiles(newPath)
      } else {
          return renamefile(file);
      }  
    });
    await Promise.all(renamePromises);
}

Answer 2

To make this work you need to wait for all the files in the directory to resolve. So you will need to do a Promise.all and use a map instead of a forEach

something like this:

try{
  await renameFiles(); // <-- await here
  console.log("do other stuff");
}
catch(){
}

const renameFiles = (path) => {
  return new Promise(resolve => {
    console.log("Renaming files...");

    const allFilesRenamePromises = fs.readdirSync(path).map(file => {
      if(file.isDirectory()) {
        let newPath = path.join(path, file);
        return renameFiles(newPath); // <- recursion here!
      } else {
        // rename file ...
      }
    }
    resolve(Promise.all(allFilesRenamePromises));
  })

Answer 3

Instead of writing one big complicated function, I'll suggest a more decomposed approach.

First we start with a files that recursively lists all files at a specified path -

const { readdir, stat } =
  require ("fs") .promises

const { join } =
  require ("path")

const files = async (path = ".") =>
  (await stat (path)) .isDirectory ()
    ? Promise
        .all
          ( (await readdir (path))
              .map (f => files (join (path, f)))
          )
        .then
          ( results =>
             [] .concat (...results)
          )
    : [ path ]

We have a way to list all files now, but we only wish to rename some of them. We'll write a generic search function to find all files that match a query -

const { basename } =
  require ("path")

const search = async (query, path = ".") =>
  (await files (path))
    .filter (x => basename (x) === query)

Now we can write your renameFiles function as a specialisation of search -

const { rename } =
  require ("fs") .promises

const { dirname } =
  require ("path")

const renameFiles = async (from = "", to = "", path = ".") =>
  Promise
    .all
      ( (await search (from, path))
          .map
            ( f =>
                rename
                  ( f
                  , join (dirname (f), to)
                  )
             )
       )

To use it, we simply call renameFiles with its expected parameters -

renameFiles ("foo", "bar", "path/to/someFolder")
  .then
    ( res => console .log ("%d files renamed", res.length)
    , console.error
    )

// 6 files renamed

---

Reviewing our programs above, we see some patterns emerging with our use of Promise.all, await, and map. Indeed these patterns can be extracted and our programs can be further simplified. Here's files and renameFiles revised to use a generic Parallel module -

const files = async (path = ".") =>
  (await stat (path)) .isDirectory ()
    ? Parallel (readdir (path))
        .flatMap (f => files (join (path, f)))
    : [ path ]

const renameFiles = (from = "", to = "", path = "") =>
  Parallel (search (from, path))
    .map
      ( f =>
          rename
            ( f
            , join (dirname (f), to)
            )
      )

The Parallel module was originally derived in this related Q&A. For additional insight and explanation, please follow the link.

Answer 4

In my first answer I showed you how to solve your problem using mainly functional techniques. In this answer, we'll see modern JavaScript features like async iterables make this kind of thing even easier -

const files = async function* (path = ".")
{ if ((await stat (path)) .isDirectory ())
    for (const f of await readdir (path))
      yield* files (join (path, f))
  else
     yield path
}

const search = async function* (query, path = ".")
{ for await (const f of files (path))
    if (query === basename (f))
      yield f
}

const renameFiles = async (from = "", to = "", path = ".") =>
{ for await (const f of search (from, path))
    await rename
      ( f
      , join (dirname (f), to)
      )
}

renameFiles ("foo", "bar", "path/to/someFolder")
  .then (_ => console .log ("done"), console.error)

Answer 5

For completeness, I'll post the entire solution based on @udalmik suggestion. The only difference is that I'm wrapping async function renameFile(file) inside a Promise.

const fs = require('fs');
const path = require('path');

const inputPath = path.resolve(process.argv[2]);
const newName = 'bar.txt';

async function renameFiles(filePath) {
    for (const file of fs.readdirSync(filePath)) {
        const newPath = path.join(filePath, file);
        const descriptor = fs.lstatSync(newPath);
        if (descriptor.isDirectory()) {
            await renameFiles(newPath)
        } else if (descriptor.isFile()) {
            await renameFile(file);
        }
    }
}

async function renameFile(file) {
  return new Promise(resolve => {
    console.log(`Renaming ${file} to ${newName}`);
    resolve();
  })
}

async function main() {
    console.log(`Renaming all files in ${inputPath} to ${newName}`);
    await renameFiles(inputPath);
    console.log('Finished');
}

main();

The reason for using the Promise is that I want to await for all the files to be renamed before continuing the execution (i.e. console.log('Finished');).

I've tested it using setTimeout

return new Promise(resolve => {
    setTimeout(()=>{
      console.log(`Renaming ${file} to ${newName}`);
    },1000)
    resolve(); // edited missing part
  })

The solution took a different path from my original question but I guess it works for me.

Answer 6

try to change the await code like this. this might help you.

try{
  const renameFilesPromise = renameFiles();
  renameFilesPromise.then({      <-- then is a callback when promise is resolved
    console.log("do other stuff");
  })
}
catch(){
}

const renameFiles = (path) => {
  return new Promise(resolve => {
    console.log("Renaming files...");

    fs.readdirSync(path).forEach(file) => {
      // if file is a directory ...
      let newPath = path.join(path, file);
      resolve( renameFiles(newPath) ); // <- recursion here!
      // else rename file ...
    }
    resolve();
  })