My requirement is to frame accureately(few frames up/down) cut on the fly mp4(h264 + aac) https video stream with ffmpeg on mobile phone fastest as I can - meaning I would trade if necessary bad compression(bigger output file) for greater cutting speed. Currently am cutting the video with this command:
ffmpeg -ss [start_time] -i https://domain/in.mp4 -t 00:00:10.000 -c:a copy -c:v copy out.mp4
This approach(bit stream copying) is very fast(less than a second on good network for 10sec long video cut) but is not accurately cutting the video. Video is being cut on key-frames which is imprecise.
So my idea is to compensate this imprecision(distance from key-frame) by seeking on video before playing it for:
delta = start_time - first_key_frame_timestamp_before_start_time
To calculate delta i need a very fast way to get time_stamp of first key-frame before start-time. I've seen post which explains how to get timestamp of first key-frame before start_time but this approach is not practical for long(2hours+) videos because it would take to long for this operation to reach lets say middle of video. I would need something like this:
ffmpeg -ss 01:30:06.000 -i https://domain/in.mp4 -t 2 -show_frames > frame_meta.txt
I have also tried doing precise cut by re-encoding the video(2hours long video, 10sec long video cut in the middle of the video) with this command:
ffmpeg -ss [start_time] -i https://domain/in.mp4 -t 10 out.mp4
but it takes to long(~16seconds) on mobile to finish compared to less than a second bit-stream copy command. Thx for any useful hint or solution.
EDIT 1:
I managed to calculate delta and get timestamp of first keyFrame before start_time by using ExoPlayer(Android multimedia player) internal classes and that approach is very fast 500ms(very short videos) up to 1500ms(for very long videos). Essentially this approach is based on parsing meta_headers(moov atom) of remote mp4 stream to found adequate timeStamp of keyFrame before cut start_time. Now NEW PROBLEM arouse, when video is cut with:
ffmpeg -ss [start_time] -i https://domain/in.mp4 -t 00:00:10.000 -c:a copy -c:v copy out.mp4
edit-list atom is added to meta header as understood from this and this post which is there to tell player to start play with delta time-offset to compensate imprecise video cutting which is done with ffmpeg bitStream copy command. But players I use on Android does not support edit-list playbacks because they just start playing normally from the beginning of video or are freezed on some frame for few moments while audio is playing and then they continue playing normally.
When I try to seek for previously calculated delta and then start play on cutted video, am not positioned on correct frame (eg. calculated delta=1 381ms but real/working delta=444ms) and my plan to compensate imprecise cutting is not working :(. If you are asking yourself if I have correctly calculated delta I've checked it with IsoViewer(mp4 container atom parser) and mannualy by stepping through frames with Avidemux and looking into time_stamp/frame_type value pair...
