Xamarin AVFoundation, add subtitles to a video

Question

I'm trying to add subtitles to a video. But no video cannot be played. Here is my code;

AVPlayer avp;
        AVPlayerViewController avpvc;
        AVMutableComposition videoPlusSubtitles;

        var subtitleEn = "SOME_VTT";
        var videoURL = "SOME_MP4_URL";
        var url = NSUrl.FromString(videoURL);

        var nsVideoUrl = new NSUrl(videoURL);
        var localVideoAsset = new AVUrlAsset(nsVideoUrl);
        videoPlusSubtitles = new AVMutableComposition();
        var videoTrack = videoPlusSubtitles.AddMutableTrack(AVMediaType.Video, 0);

        NSError error = new NSError();
        videoTrack.InsertTimeRange(CoreMedia.CMTimeRange.Zero, videoPlusSubtitles.TracksWithMediaType(AVMediaType.Video)[0], localVideoAsset.Duration, out error);

        var subtitle1 = new NSUrl(subtitleEn);
        var subtitleAsset = new AVUrlAsset(subtitle1);

        var subttileTrack = videoPlusSubtitles.AddMutableTrack(AVMediaType.Text, 0);
        subttileTrack.InsertTimeRange(CoreMedia.CMTimeRange.Zero, videoPlusSubtitles.TracksWithMediaType(AVMediaType.Text)[0], subtitleAsset.Duration, out error);

        var playerItems = new AVPlayerItem(videoPlusSubtitles);

        avpvc = new AVPlayerViewController();
        avp = new AVPlayer(playerItems);
        avpvc.Player = avp;
        AddChildViewController(avpvc);
        View.AddSubview(avpvc.View);
        avpvc.View.Frame = View.Frame;
        avpvc.ShowsPlaybackControls = true;


        avp.Play();

When I type avp = new AVPlayer(url); instead of avp = new AVPlayer(playerItems); everything works fine, so no problems with the video. Any ideas?

Could you try the code in this link :https://stackoverflow.com/questions/11525342/subtitles-for-avplayer-mpmovieplayercontroller? — ColeX, Apr 13 '21 at 13:02

score 1 · Answer 1 · answered Jan 21 '22 at 08:15

This is working for me:

AVMutableComposition videoWithSubtitlesComposition = new AVMutableComposition();
AVMutableCompositionTrack videoTrack = videoWithSubtitlesComposition.AddMutableTrack(AVMediaType.Video, 0);
AVMutableCompositionTrack audioTrack = videoWithSubtitlesComposition.AddMutableTrack(AVMediaType.Audio, 0);
AVMutableCompositionTrack subtitlesTrack = videoWithSubtitlesComposition.AddMutableTrack(AVMediaType.Text, 0);

videoTrack.InsertTimeRange(
    new CoreMedia.CMTimeRange { Start = CoreMedia.CMTime.Zero, Duration = videoAsset.Duration },
    videoAsset.GetTracks(AVMediaTypes.Video)[0],
    CoreMedia.CMTime.Zero, out _
);

audioTrack.InsertTimeRange(
    new CoreMedia.CMTimeRange { Start = CoreMedia.CMTime.Zero, Duration = videoAsset.Duration },
    videoAsset.GetTracks(AVMediaTypes.Audio)[0],
    CoreMedia.CMTime.Zero, out _
);

subtitlesTrack.InsertTimeRange(
    new CoreMedia.CMTimeRange { Start = CoreMedia.CMTime.Zero, Duration = subtitlesAsset.Duration },
    subtitlesAsset.GetTracks(AVMediaTypes.Text)[0],
    CoreMedia.CMTime.Zero, out _
);

In my case, I receive the audio and video track from an url that downloads an mp4 file and another url that downloads a vtt file. They correspond to the videoAsset and subtitlesAsset, respectively.

AVMediaType enum has "Subtitles" but using that instead of "Text" it does not work. Perhaps is a iOS bug.

Xamarin AVFoundation, add subtitles to a video

1 Answers1