Javascript (Node) Pushing Object onto Array

Question

I'm playing with Node and have a strange problem (well, strange to me). It's been a long time since I did any Javascript, so the problem is likely staring me in the face.

I'm iterating through a list of JSON text files in a directory, parsing the text from each file. It moves through the directory correctly; when I check with console.log, each object appears correct.

However, when I attempt to push this onto an array, nothing happens, the size remains one at the end of the loop. It feels like a scope issue.

Thanks for any advice.

app.get("/people/:value1/metrics/:value2", function(req, res) {
   var value1 = req.params.value1;
   var value2 = req.params.value2;
   var personPath = "people/" + value1 + "/" + value2;
   var content;
   var data = [];

   fs.readdir( personPath, function(err, files) {
     if(err) {
        console.log(err);
     }
     files.forEach( function (file, index ) {
       content = fs.readFileSync(personPath + '/' + file);
       console.log(JSON.parse(content));  //Correctly outputs the object.
       content = JSON.parse(content);
       data.push(content);
     });
   });
   console.log(data.length);  //Always 0.
   res.send(data);
});

This looks like asynchronous code, move your send into the readdir — Paul S., Dec 24 '16 at 21:54

score 1 · Answer 1 · answered Dec 24 '16 at 22:03

As it is asynchronous function code like this:

 files.forEach( function (file, index ) {
       content = fs.readFileSync(personPath + '/' + file);
       console.log(JSON.parse(content));  //Correctly outputs the object.
       content = JSON.parse(content);
       callback(content);     
   });

callback(content)
{
   data.push(content);
}

score 0 · Answer 2 · answered Dec 24 '16 at 21:51

0

You need to learn about async execution, and put this block:

console.log(data.length);
res.send(data);

right before the last bracket in fs.readdir

answered Dec 24 '16 at 21:51

Ryan

14,392
8
62
102

score 0 · Accepted Answer · edited May 23 '17 at 12:16

fs.readdir is an asynchronous execution. Your console.log(data.length) is executing before the fs.readdir execution has completed.

This also means your res.send is executing prematurely as well.

Executing Synchronously

Consider using fs.readdirSync if you'd like to keep the order of operations in your code the same:

app.get("/people/:value1/metrics/:value2", function(req, res) {
   var value1 = req.params.value1;
   var value2 = req.params.value2;
   var personPath = "people/" + value1 + "/" + value2;
   var content;
   var data = [];

   fs.readdirSync( personPath ).forEach(function(file, index) {
       content = fs.readFileSync(personPath + '/' + file);
       console.log(JSON.parse(content));  //Correctly outputs the object.
       content = JSON.parse(content);
       data.push(content); //I'm sure this is push references that are destroyed when out of scope.
   });
   console.log(data.length);  //Always 0.
   res.send(data);
});

More on fs.readdirSync here: https://nodejs.org/api/fs.html#fs_fs_readdirsync_path_options

Executing Asynchronously

You should take this time to learn more about asynchronous execution. This post has a great explanation: How does Asynchronous Javascript Execution happen? and when not to use return statement?

If you move your console.log and res.send within the scope of the fs.readdir, this should solve your problem:

app.get("/people/:value1/metrics/:value2", function(req, res) {
   var value1 = req.params.value1;
   var value2 = req.params.value2;
   var personPath = "people/" + value1 + "/" + value2;
   var content;
   var data = [];

   fs.readdir( personPath, function(err, files) {
     if(err) {
        console.log(err);
     }
     files.forEach( function (file, index ) {
       content = fs.readFileSync(personPath + '/' + file);
       console.log(JSON.parse(content));
       content = JSON.parse(content);
       data.push(content);
     });
     console.log(data.length);  //placed within the scope of fs.readdir
     res.send(data);
   });
});

EDIT: One other thing to note is that .forEach is not asynchronous. Syntactically, both fs.readdir and .forEach are similar but .forEach is a blocking method. This means that every iteration will complete before the next line executes. So moving your console.log and res.send right after the .forEach should yield the desired results.

More on .forEach being a blocking method here: JavaScript, Node.js: is Array.forEach asynchronous?

Javascript (Node) Pushing Object onto Array

3 Answers3