6

I've been thinking about this a bit, and I can't seem to come up with a reasonable solution about how to accomplish this. The problem statement is simple - write a generator that will lazily paginate a remote data set. To simplify things, this is ideally what I'd like for the user of my library to see:

for (var user of users()) { 
  console.log(user); 
}

I can't quite seem to get a generator working. I was thinking logic like this would work, but I can't figure out how to implement it.

function* users() {
  while (canPaginate) {
    yield* getNextPageOfUsers() // This will need to return an array of users from an http request
  }
}

I'm sure that I'm thinking about something wrong here, but I can't seem to find any examples of someone using a generator like this (mostly people using them with static data or people doing something like async(function*(){...}) which isn't exactly what I'm looking to do). The important part here is that I want the end user to be able to consume the data as described above.

-Vince

vincentjames501
  • 1,018
  • 11
  • 15
  • I recommend clarifying the question. In a comment on my now-deleted answer you said *"My problem isn't the consumption, it's the implementation of the generator that consumes async data"* I don't believe you *can* have the implementation of the generator be async: When you call a generator, it has to return the next item or nothing, and so can't return an async value just like normal functions can't return an async value. – T.J. Crowder Jan 09 '15 at 17:57
  • It's possible with an async generator - those are in the spec now. – Benjamin Gruenbaum Jan 09 '15 at 17:58
  • Side note: There's definitely a problem with the consumption code as well: It's looping through one page of users. To loop through multiple pages, you'd need a nested loop (one for pages, one for users within that page). – T.J. Crowder Jan 09 '15 at 17:58
  • @BenjaminGruenbaum: Cool! Where? I'm not seeing "asynchronous" anywhere but in relation to promises (in [the HTML version](https://people.mozilla.org/~jorendorff/es6-draft.html), updated Dec 24). – T.J. Crowder Jan 09 '15 at 18:00
  • 1
    Your problem is recognized and for this reason [async generators](https://github.com/jhusain/asyncgenerator) will be added to ECMASCript 7 - the peril is that no implementation supports it yet but you can [use a transpiler like 6to5](http://6to5.org/) to use it: – Benjamin Gruenbaum Jan 09 '15 at 18:01
  • @T.J.Crowder ES7 not ES6 :) Still got a while go go - I asked a related question in C# a while ago btw -http://stackoverflow.com/questions/24227235/can-i-awaitan-enumerable-i-create-with-a-generator – Benjamin Gruenbaum Jan 09 '15 at 18:01
  • @BenjaminGruenbaum: Ah, okay. "In the spec," to me, means "in the draft ES6 spec." (Or, of course, in the current spec. :-) ) – T.J. Crowder Jan 09 '15 at 18:02
  • As a functional counterpart to the imperative generator or async function - consider using an [Observable](https://github.com/Reactive-Extensions/RxJS) – Benjamin Gruenbaum Jan 09 '15 at 18:04
  • @BenjaminGruenbaum: Seems like the answer to this question is "You can't do that yet, but you'll be able to do something very similar in ES7" ("very similar" since the consumption syntax *won't* be quite like the OP wants) -- you'd be the one to post it, I'd say, having provided the most useful info. – T.J. Crowder Jan 09 '15 at 18:05
  • @T.J.Crowder I have to go eat dinner, you've very welcome to go ahead and post it - there is relevant documentation on the spec page I linked to, you can play with it on the 6to5 repl and there is additional info on the problem in https://github.com/kriskowal/gtor . Cheers. – Benjamin Gruenbaum Jan 09 '15 at 18:06
  • Indeed, definitely seems like the answer is that it's not possible currently in ES6. @T.J.Crowder, For the consumption code, ```yield*``` instead of ```yield``` should yield to another generator which would return the page's individual items (at least that's what I was thinking). – vincentjames501 Jan 09 '15 at 18:10

1 Answers1

5

Generators are functions that, in effect, pause and yield back to their callers. But when called, they must synchronously either yield back a value or complete. So they can't return the result of an asynchronous operation for the same reason that normal functions can't return the result of an asynchronous operation.

As Benjamin pointed out, there's an ES7 proposal for asynchronous generators that would let them do that, but that's ES7, and so that's markedly in the future at this point. The consumption syntax is also affected (understandably; it's important for the people writing the call to know when something is going async, we can't have normal functions look synchronous when they aren't).

According to the current proposal, your code using async generators would look something like:

for (var user on users()) { 
  console.log(user); 
}

(Note the on instead of in or of.) But that may well change.

T.J. Crowder
  • 1,031,962
  • 187
  • 1,923
  • 1,875