0

I'm trying to extract the frames from a video that is uploaded to my Vue application. Here's the function I'm working with at the moment:

async extractFramesFromVideo(video, duration) {
      return new Promise(async (resolve) => {
        await video.play();
        let seekResolve;
        video.addEventListener("seeked", async function() {
          if (seekResolve) seekResolve();
        });

        let canvas = document.createElement("canvas");
        let context = canvas.getContext("2d");
        let [w, h] = [video.videoWidth, video.videoHeight];
        canvas.width = w;
        canvas.height = h;

        let frames = [];
        let interval = 0.1;
        let currentTime = 0;

        while (currentTime < duration) {
          video.currentTime = currentTime;
          await new Promise((r) => (seekResolve = r));
          context.drawImage(this, 0, 0, w, h);
          frames = [...frames, canvas.toDataURL("image/png")];
          currentTime += interval;
        }
        resolve(frames);
      });
    }

extractFrames is called from within loadMedia:

async loadMedia(media) {
      // Create URL for uploaded video
      const newVideoSrc = URL.createObjectURL(media[0]);
      const duration = await getBlobDuration(media[0]);

      this.currentVideo.videoSrc = newVideoSrc;
      this.currentVideo.name = media[0].name;
      this.currentVideo.mimeType = media[0].type;
      this.currentVideo.duration = duration;

      // Grab video element from main workspace
      const baseVideo = document.getElementById("baseVideo");

      // Extract frames from video
      this.currentVideo.frames = this.extractFrames(baseVideo, duration);

      // Add video to Vuex store
      this.addVideoToStudio(this.currentVideo);

      // Reset currentVideo
      this.currentVideo = {
        videoSrc: "",
        mimeType: "",
        name: "",
        duration: 0,
        frames: [],
      };

      this.videoLoaded = true;
    }

With this configuration I can see the frames being added to the array, but nothing is returned. This is true if I call the method using:

this.currentVideo.frames = await this.extractFrames(baseVideo, duration);

Or using .then():

await this.extractFrames(baseVideo, duration)
  .then((frames) => {
    this.currentVideo.frames = frames;
  })
  .catch((err) => {
    console.log(err);
  });

I've logged the value of frames both inside and outside of the loadeddata event listener and frames is correct inside the event listener. However if I move resolve() inside the event listener the method doesn't run. I've tried to also define frames inside the loadeddata event listener, but that hasn't seemed to make a difference.

However, this.currentVideo.frames is not being updated properly. I have a Vuex action to add the video to my Vuex store, and it seems that I have a timing issue. I think the action is being dispatched before the frames are returned.

6/30/22 Edit: I was looking through the question @Kaiido linked in the comments, but I haven't been able to resolve my issue.

How can I get the extractFrames function to work in the way that I expect?

cpppatrick
  • 609
  • 3
  • 12
  • 29
  • "This answer has a lot of similar code, except for this part:" and `return new Promise(async (resolve) => {` maybe? The code excerpt does wait that the video actually did seek to the next portion. Otherwise (what your code is doing), you'd actually draw a single frame, because seeking a video is async. – Kaiido Jun 30 '22 at 06:12
  • @Kaiido Thank you for pointing that out. I've added that in that bit but I'm still having trouble. I've been looking through the answers that you posted and I'm not seeing a solution to my problem. I appreciate you pointing out that the seeking is also async. That provides a bit of clarity – cpppatrick Jul 01 '22 at 02:48
  • I mean, you have [a working version](https://stackoverflow.com/a/52357595/3702797) already, why can't you start from there again? Apparently all you want is to make it stop after a fixed duration. For this all you have to do is remove the line `let duration = video.duration;` (you can also remove the first while loop since you don't read the video's duration), and set `duration` as a parameter for the function instead. – Kaiido Jul 01 '22 at 03:10
  • @Kaiido I'm trying to do that. But following the directions you provided leaves me with a function that doesn't execute in my Vue app. I'm having trouble figuring out how things change when I'm passing in a video element rather than generating it in the function. Because the linked example doesn't have a `loadeddata` event to fire when the first frame is loaded. Is there another event that I can use? – cpppatrick Jul 01 '22 at 04:08
  • Just `await video.play()` – Kaiido Jul 01 '22 at 05:55
  • @Kaiido I tried to add `video.play()`in and I'm still running into problems. I edited the question to reflect my current code. I don't mean to be dense I've just never worked with video in this capacity before. So I just have so many questions and blanks I need to fill in. Thank you for your help. – cpppatrick Jul 01 '22 at 17:53

0 Answers0