Is there an ExoPlayer + Leanback library example for using captions?

Question

I have found a few examples that work with Leanback and ExoPlayer and I have all that working but I can't get subtitles/captions to work. The newest Google example I could find (https://github.com/android/tv-samples) has a captions button on the Java sample but they never show up. The Kotlin example has a comment that says // TODO(owahltinez): handle captions.

I've tried these changes to one of the samples but it didn't help:

private void prepareMediaForPlaying(Uri mediaSourceUri) {
        String userAgent = Util.getUserAgent(getActivity(), "VideoPlayerGlue");
        DefaultDataSourceFactory defaultDataSourceFactory = new DefaultDataSourceFactory(getActivity(), userAgent);
        MediaSource mediaSource =
                new ExtractorMediaSource(
                        mediaSourceUri,
                        defaultDataSourceFactory,
                        new DefaultExtractorsFactory(),
                        null,
                        null);
        String subtitle = "https://subtitledomain/sintel-en.vtt";
        Uri uriSubtitle = Uri.parse(subtitle);
        MediaSource subtitleMediaSource = new SingleSampleMediaSource.Factory(defaultDataSourceFactory)
                .createMediaSource(uriSubtitle, Format.createTextSampleFormat(null, MimeTypes.TEXT_VTT, C.SELECTION_FLAG_FORCED, "n/a"), C.TIME_UNSET);
        mediaSource = new MergingMediaSource(mediaSource, subtitleMediaSource);
        mPlayer.prepare(mediaSource);
    }

And also this change:

    mTrackSelector = new DefaultTrackSelector(videoTrackSelectionFactory);
    DefaultTrackSelector.Parameters parameters = mTrackSelector.getParameters();
    mTrackSelector.setParameters(parameters.withSelectUndeterminedTextLanguage(true));

I've tried changing the language on the subtitle to EN and that didn't help. I feel like I'm probably just missing something small but I just don't know what it could be.

Thanks.

Edit: I made a branch and removed all Leanback code and just left ExoPlayer stuff untouched and used com.google.android.exoplayer2.ui.PlayerView in my Fragment instead of VideoFragment and subtitles worked without making any other change. So it is like I just need to enable them on the Leanback side somehow.

score 3 · Accepted Answer · edited Aug 24 '20 at 02:51

3

To get captions working you need to override lb_playback_fragment.xml and add a SubtitleView.

app/src/main/res/layout/lb_playback_fragment.xml

<?xml version="1.0" encoding="utf-8"?> 
<!--
    Copied from android source so that we can add the subtitle view
-->
<FrameLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/playback_fragment_root"
    android:layout_width="match_parent"
    android:transitionGroup="false"
    android:layout_height="match_parent">

    <androidx.leanback.widget.NonOverlappingFrameLayout
        android:id="@+id/playback_fragment_background"
        android:transitionGroup="false"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

    <androidx.leanback.widget.NonOverlappingFrameLayout
        android:id="@+id/playback_controls_dock"
        android:transitionGroup="true"
        android:layout_height="match_parent"
        android:layout_width="match_parent"/>

    <com.google.android.exoplayer2.ui.AspectRatioFrameLayout
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:layout_gravity="center">

        <com.google.android.exoplayer2.ui.SubtitleView
            android:id="@+id/leanback_subtitles"
            android:layout_width="match_parent"
            android:layout_height="match_parent" />

    </com.google.android.exoplayer2.ui.AspectRatioFrameLayout>

</FrameLayout>

Once this is done you can set up a DefaultTrackSelector to select the text track you want.

You can see the full code, based on the sample leanback player, in https://github.com/bennettpeter/android-MythTV-Leanfront. The change to add subtitles is in this commit.

edited Aug 24 '20 at 02:51

Saleh

1,819
1
17
44

answered Dec 28 '19 at 19:51

Peter Bennett

64
3

1

I cannot do that... i got error Cannot resolve androidx.leanback.widget.NonOverlappingFrameLayout because it is marked as @RestrictTo(LIBRARY_GROUP) and can only be called from within the same library group... – Paweł Kanarek Sep 27 '21 at 13:28
I have been using this for a couple of years and have not seen that problem. Does that error occur at compile time or run time? Perhaps you are using a different version from me? – Peter Bennett Sep 28 '21 at 14:51
this is happening in compile time. Im using latest stable release of leanback 1.0.0... i know there are 1.1rc and 1.2alpha, but they are not official releases so my company don't use them. And exoplayer 2.14.0. – Paweł Kanarek Sep 29 '21 at 12:09
And i've managed to insert SubtitleView programatically into this view, without need of overriding lb_playback_fragment.xml. i did it by creating my own new derived class of PlaybackSupportFragment and i created new SubtitleView(requireContext()); added this to root.addView(mSubtitleView, 0); and finally had to add this line after exo intializations " val textComponent = exoplayer?.textComponent textComponent?.addTextOutput(subtitleView);" – Paweł Kanarek Sep 29 '21 at 12:14
If your error is reported against java code you may be able to prevent the error by adding @SuppressLint("RestrictedApi") in front of the function that is getting the error. – Peter Bennett Sep 30 '21 at 13:28
The error is within .xml files, so i cannot add any annotations. – Paweł Kanarek Oct 05 '21 at 09:09
I am also not able to override because of same problem – Meenakshi Khandelwal Jul 12 '23 at 10:39

score 2 · Answer 2 · answered Dec 09 '19 at 19:53

Here is another approach--renderIndex = 0 for video, =1 for audio, =2 for subtitles/text.

TrackGroupArray trackGroups = mappedTrackInfo.getTrackGroups(rendererIndex);   

TrackSelectionArray currentTrackGroups = player.getCurrentTrackSelections();
TrackSelection currentTrackSelection = currentTrackGroups.get(rendererIndex);


for (int groupIndex = 0; groupIndex < trackGroups.length; groupIndex++) {

    TrackGroup group = trackGroups.get(groupIndex);

    for (int trackIndex = 0; trackIndex < group.length; trackIndex++) {
        Format trackFormat = group.getFormat(trackIndex);


        if(currentTrackSelection!=null && 
          currentTrackSelection.getSelectedFormat()==trackFormat){
            //THIS ONE IS SELECTED
        }

    }
}

My issue isn't when setting the subtitles, if I take `Leanback` out then the subtitles work fine. I think the issue is telling the `Leanback` library to show the subtitle. — casolorz, Dec 12 '19 at 00:19

Is there an ExoPlayer + Leanback library example for using captions?

2 Answers2