117

FFmpeg can capture images from videos that can be used as thumbnails to represent the video. Most common ways of doing that are captured in the FFmpeg Wiki.

But, I don't want to pick random frames at some intervals. I found some options using filters on FFmpeg to capture scene changes:

The filter thumbnail tries to find the most representative frames in the video:

ffmpeg -i input.mp4 -vf  "thumbnail,scale=640:360" -frames:v 1 thumb.png

and the following command selects only frames that have more than 40% of changes compared to previous (and so probably are scene changes) and generates a sequence of 5 PNGs.

ffmpeg -i input.mp4 -vf  "select=gt(scene\,0.4),scale=640:360" -frames:v 5 thumb%03d.png

Info credit for the above commands to Fabio Sonnati. The second one seemed better as I could get n images and pick the best. I tried it and it generated the same image 5 times.

Some more investigation led me to:

ffmpeg -i input.mp4 -vf "select=gt(scene\,0.5)" -frames:v 5 -vsync vfr  out%02d.png

-vsync vfr ensures that you get different images. This still always picks the first frame of the video, in most cases the first frame is credits/logo and not meaningful, so I added a -ss 3 to discard first 3 seconds of the video.

My final command looks like this:

ffmpeg -ss 3 -i input.mp4 -vf "select=gt(scene\,0.5)" -frames:v 5 -vsync vfr out%02d.jpg

This was the best I could do. I have noticed that since I pick only 5 videos , all of them are mostly from beginning of the video and may miss out on important scenes that occur later in the video

I would like to pick your brains for any other better options.

d33pika
  • 1,981

9 Answers9

38

How about looking for, ideally, the first >40%-change frame within each of 5 time spans, where the time spans are the 1st, 2nd, 3rd, 4th, and 5th 20% of the video.

You could also split it into 6 time spans and disregard the 1st one to avoid credits.

In practice, this would mean setting the fps to a low number while applying your scene change check and your argument to throw out the first bit of the video.

...something like:

ffmpeg -ss 3 -i input.mp4 -vf "select=gt(scene\,0.4)" -frames:v 5 -vsync vfr -vf fps=fps=1/600 out%02d.jpg
Dej
  • 213
A.M.
  • 1,017
  • 5
  • 17
  • 24
18

Defining meaningful is hard but if you want to make N thumbnails efficiently spanning whole video file this is what I use to generate thumbnails on production with user uploaded content.

Pseudo-code

for X in 1..N
  T = integer( (X - 0.5) * D / N )  
  run `ffmpeg -ss <T> -i <movie>
              -vf select="eq(pict_type\,I)" -vframes 1 image<X>.jpg`

Where:

  • D - video duration read from ffmpeg -i <movie> alone or ffprobe which has nice JSON output writer btw
  • N - total number of thumbnails you want
  • X - thumbnail number, from 1 to N
  • T - time point for tumbnail

Simply the above writes down center key-frame of each partition of the movie. E.g. if movie is 300s long and you want 3 thumbnails then it takes one key frame after 50s, 150s and 250s. For 5 thumbnails it would be 30s, 90s, 150s, 210s, 270s. You can adjust N depending on movie duration D, that e.g. 5 minute movie will have 3 thumbnails but over 1 hour will have 20 thumbnails.

Performance

Each invocation of above ffmpeg command takes a fraction of second (!) for ~1GB H.264. That is because it instantly jumps to <time> position (mind -ss before -i) and takes first key frame which is practically complete JPEG. There is no time wasted for rendering the movie to match exact time position.

Post-processing

You can mix above with scale or any other resize method. You can also remove solid color frames or try to mix it with other filters like thumbnail.

gertas
  • 281
  • 2
  • 4
6

Try this

 ffmpeg -i input.mp4 -vf fps= no_of_thumbs_req/total_video_time out%d.png

Using this command I am able to generate the required number of thumbnails which are representative of the entire video.

smali
  • 738
LostPuppy
  • 161
3

This is a really hard task to do. The very first thing to do is define "meaningful" in your context: are you looking for the most uniform image? an image that describe your video but does not spoil it?

For most of the cases, the best thumbnail is the one that has:

  • Great Luminance.
  • Does not have a high uniformity of Pixel Intensity.
  • Good Contrast.
  • Great Sharpness.

Using ffmpeg, that means removing black frames and then get a thumbnail (ffmpeg has a nice filter to get thumbnails). First get a list of black frames:

ffprobe -f lavfi -i "movie=./input.mp4,blackdetect[out0]" -show_entries tags=lavfi.black_start,lavfi.black_end -of default=nw=1 -v quiet

then, remove those from the video and get the thubmnail:

ffmpeg -i ./input.mp4 -filter_complex "[0:v]trim=start=3.23657:end=32.9329,setpts=PTS-STARTPTS[v1];[0:a]atrim=start=3.23657:end=32.9329,asetpts=PTS-STARTPTS[a1]" -map [v1] -map [a1] output.mp4

and

ffmpeg -i output.mp4 -vf "thumbnail" -frames:v 1 thumb2.png

You will notice that the output is not the best that you can get, this method only pick a frame that has some "content" and does not consider the factors I described before.

We wrote a blog post describing this very same problem that might be useful.

Disclaimer: I'm one of the co-founders of https://mediamachine.io

GiaNU
  • 131
2

Here's what I do to generate a periodic thumbnail for live m3u8 streams to use as a poster. I found running a continuous ffmpeg task just to generate thumbnails eats up all my CPU, so instead I run a cronjob every 60 seconds that generates all the thumbnails for my streams.

#!/usr/bin/env bash

## this is slow but gives better thumbnails (takes about 1+ minutes for 20 files)
#-vf thumbnail,scale=640:360 -frames:v 1 poster.png

## this is faster but thumbnails are random (takes about 8 seconds for 20 files)
#-vf scale=640:360 -frames:v 1 poster.png
cd ~/path/to/streams

find . -type f \
  -name "*.m3u8" \
  -execdir sh -c 'ffmpeg -y -i "$0" -vf scale=640:360 -frames:v 1 "poster.png"' \
  {} \;

If you need to do more frequently than 60 seconds (limitation of cronjob), then you can do a while loop in your script and it'll execute forever. Just add a sleep 30 to change the frequency to 30 seconds. I don't recommend doing this with large number of videos though, as the previous run may not complete before the next one starts.

Ideally with cronjob I just run it every 5 minutes.

chovy
  • 1,195
2

I once did something similar, but I exported all frames of the video (in 1 fps) and compared them with a perl utility I found which computes the difference between images. I compared each frame to previous thumbnails, and if it was different from all thumbnails, I added it to the thumbnails collection. The advantage here is that if your video moves from scene A to B and them returns to A, ffmpeg will export 2 frames of A.

Eran Ben-Natan
  • 374
1

I can't add comments yet, so in reference to @jaygooby solution causing a syntax error Meaningful thumbnails for a Video using FFmpeg, the problem is that $N (number of screenshots) wasn't declared. The fix is to add it at the beginning of the command, like this:

N=30; D=180; for X in $(seq 1 $N); do echo $X; T=$(bc <<< "($X-0.5)*$D/$N"); ffmpeg -y -hide_banner -loglevel panic -ss $T -i in.mp4 -vf select="eq(pict_type\,I)" -vframes 1 $X.jpg; done

Furthermore, @jaygooby's solution works well and is very fast (because of putting -ss before -i in ffmpeg's command). I combined both of his suggestions (the one with mediainfo) plus my fix:

N=60; INPUT="path/to/filename"; D=$(bc <<< $(mediainfo --Inform="Video;%Duration%" "$INPUT")/1000) ; echo "Duration: $D seconds"; for X in $(seq 1 $N); do echo $X; T=$(bc <<< "($X-0.5)*$D/$N"); ffmpeg -y -hide_banner -loglevel panic -ss $T -i $INPUT -vf select="eq(pict_type\,I)" -vframes 1 $X.jpg; done

where:

N = number of desired screenshots
INPUT = path to the filename

Change those two variables and you're good to go.

synkro
  • 154
0

https://github.com/romanwarlock/thumbnails/blob/master/thumbgen.sh

Usage:

. thumbgen.sh COLUMNS ROWS SIZE INPUT

COLUMNS means number of columns; ROWS means number of rows; SIZE is the length of the longer side of the output, e.g., 1920 if you want to get an 1920x1080 output image; INPUT is the path to the input file;

Example:

. thumbgen.sh 3 4 1920 video.mp4

. <path-to-file>/thumbgen.sh 3 4 1920 video.mp4

output example: https://github.com/romanwarlock/thumbnails/blob/master/th7138.jpg

in the script all the screenshots are stored in TMPDIR=/tmp/thumbnails-${RANDOM}/ but the folder can be reset of course.

oktobris
  • 11
  • 1
  • 1
  • 4
0

Definitely depends also on the video you want to get the thumbs from. For a classical slides lecture with a talking head in the corner, my resulty were best with ffmpeg -i vid.mp4 -vf "thumbnail,select=gt(scene\,0.015)" -vsync vfr -r 1 -frame_pts 1 out%010d.jpg -- so a combination of the built-in thumbnail extractor, and the look for a small change in the video scene. Some slide animations were taken superfluously, but also I could be sure to not miss anything.

The -r 1 -frame_pts 1 out%010d.jpg part just results in files named after the second of the video, in which they occurred.

Jaleks
  • 277
  • 2
  • 12