How to wait until all async calls are finished

Question

I've got NestJS application which interact with YoutubeAPI and load videos from it. One particular method is important and it's loadVideos from below. Method it self has multiple asyncs inside and I need to work with videoIdMap property once everything is finished

private loadVideos(
    playListId: string,
    channel: Channel,
    nextPageToken: string,
    stopLoadingOnVideoId: string,
  ) {
    const baseUrl = YoutubeService.VIDEO_URL_SNIPPET_BY_ID + playListId;
    const response = this.httpService
      .get(nextPageToken ? baseUrl + '&pageToken=' + nextPageToken : baseUrl)
      .pipe(map((response) => response.data));
    response.subscribe((data) => {
      data.items.forEach((item) => {
        if (stopLoadingOnVideoId && item.snippet.resourceId.videoId === stopLoadingOnVideoId) {
          return;
        }        
        this.prepareVideoEntity(item.snippet, channel).then((partialVideo) =>              
          this.videoService.create(partialVideo).then((video) => {     
            this.videoIdMap[video.youtubeId] = video.id;
          }),
        );
      });      
      if (data.nextPageToken) {        
        this.loadVideos(
          playListId,
          channel,
          data.nextPageToken,
          stopLoadingOnVideoId,
        );
      }
    });
  }

Ideal solution for me would be to make loadVideos async somehow so I can later do:

public methodWhichCallLoadVideos(): void {
  await loadVideos(playListId, channel, null, stopLoadingOnVideoId)
  // My code which have to be executed right after videos are loaded
}

Every solution I tried out end up with this.videoIdMap to be empty object or with compilation issue so any idea is more than welcome.

In order to "await" loadVideos, you need to make it "async". For that, it has to return a promise while Nest's HttpService returns an observable. Check this question: https://stackoverflow.com/questions/51910908/nestjs-async-httpservice-call — Max Ivanov, Oct 15 '20 at 16:11

score 2 · Accepted Answer · answered Oct 16 '20 at 06:35

You could switch to promises instead of Observables, thus turning the method into an async one that recurs as long as data has a nextPageToken:

private async loadVideos(
        playListId: string,
        channel: Channel,
        nextPageToken: string,
        stopLoadingOnVideoId: string,
    ) {
        const baseUrl = YoutubeService.VIDEO_URL_SNIPPET_BY_ID + playListId;
        const response = await this.httpService
            .get(nextPageToken ? url + '&pageToken=' + nextPageToken : url).toPromise();
        const { data } = response;
        for (const item of data.items) {
            if (stopLoadingOnVideoId && item.snippet.resourceId.videoId === stopLoadingOnVideoId) {
                continue;
            }
            const partialVideo = await this.prepareVideoEntity(item.snippet, channel);
            const video = await this.videoService.create(partialVideo)
            this.videoIdMap[video.youtubeId] = video.id;
        }
        if (data.nextPageToken) {
            await this.loadVideos(
                playListId,
                channel,
                data.nextPageToken,
                stopLoadingOnVideoId,
            );
        }
    }

In your caller you can then simply await loadVideos(...):

private async initVideoIdMap(...) {
  await this.loadVideos(...);
  // this.videoIdMap should be correctly populated at this point
}

Accepted + THUMBS UP! This is nice and clean code I was looking for. Can I only ask why you replaced `forEach` with `for` loop? — Andurit, Oct 16 '20 at 09:42
Happy to help - `foreach` will not await the async calls that are done inside the loop. It will just fire off and not wait for the promises to resolve. Here's an article that explains it in detail: https://itnext.io/why-async-await-in-a-foreach-is-not-working-5f13118f90d — eol, Oct 16 '20 at 09:45

How to wait until all async calls are finished

1 Answers1