Why do the NodeJS shell and compiler disagree on assignments to variables?

Question

The following are the contents of the file tags.js.

//the following initializes the MongoDB library for Javascript.
var MongoClient = require('mongodb').MongoClient,
assert = require('assert');

//connection URL
abc = 'mongodb://localhost:27017/abcbase/

/* the following is a global variable
    where I want retrieved content */
var arrayToReturn = []; 

/* toFind is a dictionary, toSearch is a 
   MongoDB collection (input is its name as a string) */
function findElemHelper(toSearch, toFind) {
    /* establishes connection to the database abcbase
       and runs the contents of the function that is
       the second input */
    MongoClient.connect(abc, function(err, db) {
        /* coll is the collection being searched.
           tempCurs is the results of the search returned
           as a cursor
        */
        coll = db.collection(toSearch);
        tempCurs = coll.find(toFind);
        /* the three lines below this comment
           form the crux of my problem. I expect
           arrayToReturn to be assigned to docs
           (the contents of the cursor received by 
           the find function). Instead it seems like
           it is declaring a new variable in local scope.*/
        tempCurs.toArray(function(err, docs) {
            arrayToReturn = docs;
        }); 
    }); 
}

function findElem(toSearch, toFind) {
    findElemHelper(toSearch, toFind);
    return arrayToReturn;
}

function returnF() {
    return arrayToReturn;
}

var ln = findElem("userCollection", {});
var lm = returnF();

console.log(ln);
console.log(lm);

When I run the file using the node interpreter with the command node tags.js, it prints

[]
[]

And when I run the same code on the node interpreter (which I enter with the command node from Terminal and copy-paste the same code into the shell), console.log(ln) prints [] and console.log(lm) prints the contents of the document I want to retrieve from MongoDB.

Could someone explain this behavior?

It's likely that you're running `returnF` before `findElem` is complete. Many actions in node.js are asynchronous, such as communicating with a database. — Kevin B, Jul 06 '15 at 15:32
possible duplicate of [How to return the response from an asynchronous call?](http://stackoverflow.com/questions/14220321/how-to-return-the-response-from-an-asynchronous-call) — Ben Fortune, Jul 06 '15 at 15:36
Don't know. Maybe the REPL simply takes longer and so the response has a chance to return. Doesn't matter so much since that's not where your code is going to run. — , Jul 06 '15 at 15:37
@BenFortune: That doesn't explain why it works differently in the REPL. Though ultimately that's what he'll need. — , Jul 06 '15 at 15:38
@squint Sounds like the OP is typing the `console.log`'s manually, which will probably be populated by the time they're finished typing. — Ben Fortune, Jul 06 '15 at 15:41
@BenFortune: That could be. I thought he was saying he did a copy/paste of the entire thing, which makes the output surprising, though a REPL could have enough delay to allow for it. Not sure. But line-by-line would definitely explain it. — , Jul 06 '15 at 15:44

score 1 · Answer 1 · edited May 23 '17 at 12:06

As some commenters pointed at, the problem is the async nature of the findElemHelper method. This long, but detailed answer posted on one of the comments explains the basics behind async in javascript and how to approach this style of coding in general.

In a shorter answer, with asynchronous code you cannot assume the order of operations is the same as statements in your code. You are correct in identifying the location of the crux of your problem, but the issue is not scope, but rather that the function you passed into tempCurs.toArray is called whenever the database returns data, which could be after the rest of the file has finished executing. (The same is true of the function passed into MongoClient.connect, where you could end up calling console.log before the db even connects!)

Here's how we solve the problem with callbacks, the goal is to structure our code such that we are certain that the database has returned the data before calling console.log:

var MongoClient = require('mongodb').MongoClient;

var abc = 'mongodb://localhost:27017/abcbase/';

/**
 * Take a callback function as the last parameter which will
 * be called when the array is retrieved.
 */
function findElem(toSearch, toFind, callback) {

  MongoClient.connect(abc, function(err, db) {

    var coll = db.collection(toSearch);
    var tempCurs = coll.find(toFind);
    tempCurs.toArray(callback);

  }); 
}

findElem("userCollection", {}, function(err, docs) {

  console.log(docs);

});

score 0 · Answer 2 · answered Jul 06 '15 at 18:38

Your function findElemHelper makes an asynchronous call to MongoDB with a callback. Thus, you don't really know when arrayToReturn has its contents populated. Chances are, printing lm in the console means you gave it enough time to actually populate.

You should try to restructure your code so that you can use the response from the asynchronous call in your callback, rather than outside in a global variable.

Why do the NodeJS shell and compiler disagree on assignments to variables?

2 Answers2