2

I want to create a system where a user can upload two videos to a website, one of which has blue screen in the background, and the website uses AVCONV (or similar) to create a composite video of the two using the Chroma Key technique (aka blue screening).

Can AVCONV be used in this way?

According to this post on SO a static image can be Chrome Keyed into the background. But can the same be done with a video?

I would also like to know if the two videos would need to be of the same length and format before they could be merged? Are there any other technical considerations I should be looking into to be able to offer a service like this?

Thanks in advance.

Community
  • 1
Jimmery
  • 630

3 Answers3

3

You can do this using MLT

melt https://archive.org/download/IMB_SF_R35_C8/IMB_SF_R35_C8_512kb.mp4 in=400 \
    -track https://archive.org/download/IMB_SF_R19_C9/IMB_SF_R19_C9_512kb.mp4 in=450 \
    -filter chroma key=0x00ff0000 variance=0.45 \
    -transition composite

This plays a background video and a green-screen clip of a dog as a second track composited on top. It uses a chroma key (set to green with some variance) to apply the alpha to the green areas allowing the composited background to show through. The in=XXX are just to truncate off some leading garbage on the video clips being used.

rectalogic
  • 131
2

Note: This answer was posted before FFmpeg gained the capability to do chroma keying. See other answers below.

Neither AVconv nor FFmpeg can currently do this - it's the one major video processing task that they cannot do.

These tools can overlay an image or video with a transparent background over another video or image - which is what that SO question you linked to wanted. However, the video with the transparent background needs to be in a video format that supports transparency (most of them don't) - I'd use qtrle (Apple's Animation codec). But they cannot do the actual chroma-keying (turning a certain colour transparent).

You could probably use a workaround involving avconv and ImageMagick, but it would be rather CPU-intensive, and I don't know how good your results would be. You would need to:

  1. Turn the video you want to chroma-key into individual frames with avconv
  2. Use ImageMagick's convert (or mogrify) tool to do the chroma-keying on each individual frame
  3. Stitch the frames back together with avconv & place it over the background video

You may be able to simplify the process using pipes.

To extract the frames (produces files that look like 00001.png, 00002.png, etc):

avconv -i input.mp4 %05d.png

To stitch them back together & put on top of your background video after you've run ImageMagick over them:

avconv -f image2 -r 25 -i %05d.png -i background.mp4 \
-filter_complex '[1:v][0:v]overlay[out]' \
-map [out] -c:v libx264 -crf 23 -preset veryfast output.mp4

-r 25 gives you a frame rate of 25; if you want another frame rate, just use a different number. Make sure you use the same frame rate as the original input video. If you want to take the audio from your original input, use

avconv -f image2 -r 25 -i %05d.png -i background.mp4 -i input.mp4 \
-filter_complex '[1:v][0:v]overlay[out]' \
-map [out] -map 2:a -c:v libx264 -crf 23 -preset veryfast -c:a copy output.mp4

Note that some versions of AVconv (like the version in the Ubuntu repos) don't have support for filtergraphs. If that's the case for you, upgrade to a newer version of avconv or ffmpeg.

As for stage 2... I'm actually not sure how to do it, but I know that ImageMagick can do chroma-keying. Here are a few links you may find useful:

http://www.imagemagick.org/Usage/photos/#chroma_key
http://www.imagemagick.org/discourse-server/viewtopic.php?t=14394
http://tech.natemurray.com/2007/12/convert-white-to-transparent.html

Due to differences in lighting etc between the samples people will give you, you'll have to use something to detect the exact shades of blue to get rid of. You'll probably have to leave that up to your users, perhaps using ffmpeg to create a thumbnail, and then using an eyedropper to choose the background colour.

slhck
  • 235,242
evilsoup
  • 14,056
1

chroma key is available in ffmpeg from version 3.0 ,before it you can use colorkey option

ffmpeg -f mpeg -i input.mp4 -i video.mp4 -shortest -filter_complex "[1:v]chromakey=0x70de77:0.1:0.2[ckout];[0:v][ckout]overlay[out]" -map "[out]" output.mp4

for colorkey in ffmpeg v2.8 or above

 ffmpeg -f mpeg -i input.mp4 -i video.mp4 -shortest -filter_complex "[1:v]colorkey=0x70de77:0.1:0.2[ckout];[0:v][ckout]overlay[out]" -map "[out]" output.mp

Both of these are working for me.

Ravikant Tiwari
  • 111