Understanding closures: Constructing a meta-function that queues functions together

Question

In terms of solving the problem, I have a fully working solution that I just finished here:

// synchronous dynamic script loading. 
// takes an array of js url's to be loaded in that specific order. 
// assembles an array of functions that are referenced more directly rather than 
// using only nested closures. I couldn't get it going with the closures and gave up on it. 

function js_load(resources, cb_done) {
    var cb_list = []; // this is not space optimal but nobody gives a damn 
    array_each(resources, function(r, i) {
        cb_list[i] = function() {
            var x = document.body.appendChild(document.createElement('script'));
            x.src = r;
            console.log("loading "+r);
            x.onload = function() { 
                console.log("js_load: loaded "+r); 
                if (i === resources.length-1) {
                    cb_done();
                } else {
                    cb_list[i+1]();
                }
            }; 
        };
    });
    cb_list[0]();
}

I am completely happy with this because it does what I want now, and is probably far easier to debug than what my first approach, if it had succeeded, would have been.

But what i can't get over is why I could never get it to work.

It looked something like this.

function js_load(resources, cb_done) {
    var cur_cont = cb_done;
    // So this is an iterative approach that makes a nested "function stack" where 
    // the inner functions are hidden inside the closures. 
    array_each_reverse(resources, function(r) {
        // the stack of callbacks must be assembled in reverse order
        var tmp_f = function() {
            var x = document.body.appendChild(document.createElement('script'));
            x.src = r;
            console.log("loading "+r);
            x.onload = function() { console.log("js_load: loaded "+r); cur_cont(); }; // TODO: get rid of this function creation once we know it works right 
        };
        cur_cont = tmp_f; // Trying here to not make the function recursive. We're generating a closure with it inside. Doesn't seem to have worked :(
    });
    cur_cont();
}

It kept trying to call itself in an infinite loop, among other strange things, and it's really hard to identify which function a function is and what a function contains within it, during debugging.

I did not dig into the code, but it appears that jQuery.queue has also implemented a similar mechanism to my working one (using an array to track the queue of continuations) rather than using only closures.

My question is this: Is it possible to build a Javascript function that can take a function as argument, and enhance it with a list of other functions, by building closures that wrap functions it creates itself?

This is really hard to describe. But I'm sure somebody has a proper theory-backed mathematical term for it.

P.S. Referenced by the code above are these routines

// iterates through array (which as you know is a hash), via a for loop over integers
// f receives args (value, index)
function array_each(arr, f) {
    var l = arr.length; // will die if you modify the array in the loop function. BEWARE
    for (var i=0; i<l; ++i) {
        f(arr[i], i);
    }
}

function array_each_reverse(arr, f) {
    var l = arr.length; // will die if you modify the array in the loop function. BEWARE
    for (var i=l-1; i>=0; --i) {
        f(arr[i], i);
    }
}

Answer 1

The problem is how you were setting the value of cur_cont for every new function you made, and calling cur_cont in the onload callback. When you make a closure like tmp_f, any free variables like cur_cont are not 'frozen' to their current values. If cur_cont is changed at all, any reference to it from within tmp_f will refer to the new, updated value. As you are constantly changing cur_cont to be the new tmp_f function you have just made, the reference to the other functions are lost. Then, when cur_cont is executed and finishes, cur_cont is called again. This is exactly the same function that had just finished executing - hence the infinite loop!

In this sort of situation, where you need to keep the value of a free variable inside a closure, the easiest thing to do is to make a new function and call that with the value you want to keep. By calling this new function, a new variable is created just for that run, which will keep the value you need.

function js_load(resources, cb_done) {
    var cur_cont = cb_done;
    array_each_reverse(resources, function(r) {
        // the stack of callbacks must be assembled in reverse order

        // Make a new function, and pass the current value of the `cur_cont`
        // variable to it, so we have the correct value in later executions.
        // Within this function, use `done` instead of `cur_cont`;
        cur_cont = (function(done) {

            // Make a new function that calls `done` when it is finished, and return it.
            // This function will become the new `cur_cont`.
            return function() {

                var x = document.body.appendChild(document.createElement('script'));
                x.src = r;
                console.log("loading "+r);
                x.onload = function() {
                    console.log("js_load: loaded "+r);
                    done();
                };
            };
        })(cur_cont);

    });

    // Start executing the function chain
    cur_cont();
}

EDIT: Actually, this can be made even simpler by using the Array.reduce function. Conceptually, you are taking an array and producing a single function from that array, and each successive function generated should be dependant upon the last function generated. This is the problem that reduce was designed to help solve:

function js_load(resources, done) {
    var queue = resources.reduceRight(function(done, r) {
        return function() {
            var x = document.body.appendChild(document.createElement('script'));
            x.src = r;
            console.log("loading "+r);
            x.onload = function() {
                console.log("js_load: loaded "+r);
                done();
            };
        };
    }, done);

    queue();
};

Note that reduce and reduceRight are not available for older browsers (<= IE8). A JavaScript implementation can be found on the MDN page.