17

My initial thought was to upload audio files to YouTube along with video that is inspired from the audio. The particular visualization can be in different form such as spectrum, spectogram, or other forms of visualizations that change with the audio. I'm not familiar with all the capabilities of ffmpeg or sox, but I wonder if I can do something like this out of the box, or as a series of scripts with other command line utilities.

Sun
  • 6,480

3 Answers3

23

Audio visualization with ffmpeg

Audio visualization with ffmpeg

ffmpeg -i input.mp3 -filter_complex \
"[0:a]avectorscope=s=640x518,pad=1280:720[vs]; \
 [0:a]showspectrum=mode=separate:color=intensity:scale=cbrt:s=640x518[ss]; \
 [0:a]showwaves=s=1280x202:mode=line[sw]; \
 [vs][ss]overlay=w[bg]; \
 [bg][sw]overlay=0:H-h,drawtext=fontfile=/usr/share/fonts/TTF/Vera.ttf:fontcolor=white:x=10:y=10:text='\"Song Title\" by Artist'[out]" \
-map "[out]" -map 0:a -c:v libx264 -preset fast -crf 18 -c:a copy output.mkv

ffmpeg can use several filters to visualize audio: avectorscope, showspectrum, and showwaves. You can then place them where you want with overlay, and then add text with drawtext.

In the example above the audio is stream copied (re-muxed) instead of being re-encoded.

From FFmpeg Wiki: How to Encode Videos for YouTube and other Video Sharing Sites.

llogan
  • 63,280
18

Here are some examples for taking an audio file, running it through ffmpeg, and have a video created based on some of the filters available in ffmpeg.

Examples:

spectogram:

ffmpeg -i song.mp3 -filter_complex showspectrum=mode=separate:color=intensity:slide=1:scale=cbrt -y -acodec copy video.mp4

spectogram

avectorscope:

ffmpeg -i song.mp3 -filter_complex avectorscope=s=320x240 -y -acodec copy video.mp4

avectorscope

zooming mandelbrot:

ffmpeg -i song.mp3 -f lavfi -i mandelbrot=s=320x240 -y -acodec copy video.mp4

(Screenshot missing)

source: [Libav-user] ffmpeg showspectrum to file

Flimm
  • 10,730
Sun
  • 6,480
4

I use this:

ffmpeg -y -i audio.mp3 -loop 1 -i image.jpg -filter_complex "[1:v]crop=640:480:0:0,setsar=1[img]; [0:a]showwaves=mode=line:s=hd480:colors=Yellow@0.5|Blue@0.5:scale=sqrt,format=yuva420p[waves]; [img][waves]overlay=format=auto,drawtext=text='${NAME}':fontcolor=White@0.5:fontsize=30:font=Arvo:x=(w-text_w)/5:y=(h-text_h)/5[out]" -map "[out]" -map 0:a -pix_fmt yuv420p -b:a 360k -r:a 44100 -c:v libx264 -q:v 23 -preset ultrafast -c:a copy -shortest out.mkv

It's a "standing wave" effect on top of an image with overlayed text (e.g. track name)

So I take a JPG image from unsplash, put in folder as "image.jpg". Then I take audio.mp3 and combine with wave effect into a 480p video. I guess you can adjust 480p to HD.

[visual effects illustration

grandrew
  • 198