Asynchronous map/reduce in Javascript/JQuery

Question

What's the idiomatic way to do the following in Javascript (jQuery)?

Spawn a set of asynchronous jobs
Collect partial results
When every job has completed, combine partial results

The above can be achieved by something on the lines of (assuming for simplicity that requests are processed in order; in a more realistic case, a counter would do the job):

var results = new Array();
$.each(objs, function(i,obj)){
  $.getJSON(make_async_request(obj), function(data) {
    results[data.key] = data.value;
    if (i == objs.length-1)
      elaborate(results);
  }
});

Which looks ugly to me. Any suggestions?

Slamming the server with multiple HTTP requests will not necessarily make things better. Ofcourse we don't know your circumstances, but your code seems fine (except the bit that checks `i==objs.length-1` because there is no gaurantee that the last request fired off will complete last). — Strelok, Jul 22 '12 at 16:01
You're absolutely right. I've updated the question with the assumption I'm making here. — Roberto Aloi, Jul 22 '12 at 21:39

David Hellsing · Accepted Answer · 2012-07-22T16:19:15.257

You can use jQuery.Deferred like this:

   $.when($.get('/a/'), $.get('/b/')).then(function() {
       // all gets are ready
   });

If you need to merge all results in an an object, you can loop the arguments in the then callback:

$.when($.get('/a/'), $.get('/b/')).then(function() {
    var results = {},
        args = Array.prototype.slice.call(arguments),
        data;
    $.each( args, function(i, resp) {
        data = resp[0];  // resp is the results array
        results[data.key] = data.value;
    });
    console.log(results);
});

score 0 · Answer 2 · answered Jul 22 '12 at 16:14

var results = [], ajaxes = [];
$.each(objs, function(i,obj)){
    var xhr = $.ajax({
        url: make_async_request(obj),
        dataType: 'json'
    }).done(function() {
        results[data.key] = data.value;
        if (i == objs.length-1) elaborate(results);
    });
    ajaxes.push(xhr);
});

$.when.apply(null, ajaxes).done(function() {
    alert('all ajaxes are done');
});

score 0 · Answer 3 · answered Jul 22 '12 at 16:20

You check on the completion of the jobs is inadequate as $.getJSON is async and returns immediately after making the request, and so as soon as the last request was made your check will evaluate to true.

You can effectively do what you want by utilizing the fact the $.ajax and therefore $.getJSON return Deferreds. Use $.when and $.map to collect the data of all requests for processing. Have a look at the deferred/promise apis in jQuery they are very powerful and allow for something like the following one-liner to work:

$.when($(objs).each(function (i, obj) { return $.getJSON(make_async_request(obj)); }), elaborate);

Could you elaborate a bit more? I would have expected a "map" in place of the "each" and a "then" before the "elaborate". The return in the each-function seems weird, as in contrast to "map" each only uses the return value to decide if you want to abort the loop. So you could remove the "return" keyword in the first place. — Falk Tandetzky, Apr 25 '21 at 12:07

score 0 · Answer 4 · 2012-07-22T22:09:07.363

Full disclosure: blatant plugging of my own library, queue-flow :)

q(objs)
    .map(q.async(function(obj, callback) {
        $.getJSON(make_async_request(obj), callback);
    }))
    .reduce(function(results, data) {
        results[data.key] = data.val;
        return results;
    }, elaborate, {});

queue-flow uses the concept of queues to make async functional code look almost perfectly synchronous. :) It also lets you do some very interesting things with source code organization, take a look at the .branch method in the readme on GitHub. (Just updated my example to match the change in the question example.)

That could also be written as:

q(objs)
    .map(make_async_request)
    .map(q.async($.getJSON))
    .reduce(function(results, data) {
        results[data.key] = data.val;
        return results;
    }, elaborate, {});

Which is really cool, but might take a bit for someone unfamiliar to grok. :)

Falk Tandetzky · Answer 5 · 2021-04-25T13:25:55.387

0

Update as of 2018: By now you can use

results = await Promise.all(objs.map(someAsyncFunction));

where objs should be some list and someAsyncFunction takes one element of this list and returns a promise with the result. For more details see the answers to this question: Best way to call an async function within map?

edited Apr 25 '21 at 13:25

answered Apr 25 '21 at 12:46

Falk Tandetzky

5,226
2
15
27

Asynchronous map/reduce in Javascript/JQuery

5 Answers5