3

I use my phone for streaming live videos to my server the usual way with javascript and getUserMedia() etc. Then I'm using FFMPEG to encode it in realtime and additionally put an overlay video onto it.

For some reason, each time I rotate the phone to landscape mode or back to portrait mode, the overlay video will jump forward in time. The overlay video is not being modified while the live stream is encoding. It's just a video file from the hard-drive, no stream. Any ideas whats going on?

I tried different FFMPEG versions, but the behaviour is always the same.

Orange is the livestream. Green is the overlay: Orange is the livestream. Green is the overlay.

Video Sample
Here is a short sample of whats going on.

Works after downloading 2mb
Sample
Works with the online Mega Player 23mb
Sample

FFMPEG Command

ffmpeg -re -i strm.webm -i overlay.webm -filter_complex "[0:v]scale=640:480:force_original_aspect_ratio=decreasepad=640:480:-1:-1:color=black[main];[1]scale=iw/1.5:-1setpts=PTS-STARTPTS,format=yuva420p[ovrl]; [main][ovrl]overlay=W-w-10:10[v]; [0:a][1:a]amerge[a]" -r 24 -deadline realtime -crf 22 -cpu-used 15 -level 2.0 -map "[v]" -map "[a]" -c:a aac -c:v libvpx -pix_fmt nv12 -threads 4 -strict -2 -auto-alt-ref 0 -b:v:1 1800k -s:v:1 1280x720 -b:v:0 300k -s:v:0 640x480 -bf 1 -keyint_min 48 -g 48 -sc_threshold 1 -b_strategy 0 -ar:a:1 96000 -scenario livestreaming -look_ahead 0 -seg_duration 2 -remove_at_exit 1 -streaming 1 -window_size 30 -adaptation_sets "id=0,streams=v id=1,streams=a" -utc_timing_url https://time.akamai.com/?iso -live 1 -y -f dash manifest.mpd
Macster
  • 41

1 Answers1

1

As @Rotem rightly pointed out, “setpts” was the problem.
For some reason, resolution change, caused by the rotation, led to new incorrect timestamps being set by “setpts” for the overlay. So I just removed the setpts option.
No setpts, no new false timestamps.

So the filter_complex simply looks like this:

-filter_complex [0:v]scale=640:480:force_original_aspect_ratio=decrease,pad=640:480:-1:-1:color=black[main];[1]scale=iw/1.5:-1,format=yuva420p[ovrl]; [main][ovrl]overlay=W-w-10:10[v]; [0:a][1:a]amerge[a]"
Macster
  • 41