2

This command produce init.mp4 + bunch of m4s files, i'm trying to play them using MSE : 

ffmpeg -i <input file> -f hls -hls_segment_type fmp4 -c:v copy playlist.m3u8

This is the client side code i'm using:

var socket = io();
var video = document.querySelector('video');
var mimeCodec = 'video/mp4; codecs="avc1.64000d,mp4a.40.2"'; 
if ('MediaSource' in window && MediaSource.isTypeSupported(mimeCodec)) {
    var mediaSource = new MediaSource;
    video.src = URL.createObjectURL(mediaSource);
    mediaSource.addEventListener('sourceopen', sourceOpen);
} else {
    console.error('Unsupported MIME type or codec: ', mimeCodec);
}
function sourceOpen (_) {
  var mediaSource = this;
  var sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);
  sourceBuffer.mode = 'sequence'; 
  socket.on('broadcast', function (newPiece) {
      // here i'm getting the buffer of the video  == buffer
      sourceBuffer.addEventListener('updateend', function (_) {
        video.play().then(function() { }).catch(function(error) { });
      });
      sourceBuffer.appendBuffer(buffer); // when the seconde video comes i append it's buffer
  })
};

Everything works fine when i send init.mp4 file followed by playlist0.m4s, playlist1.m4s, playlist2.m4s, ..... But when i try to play init.mp4 file followed immediately 6,7,8 not 0,1,2 meaning playlist6.m4s, playlist7.m4s, playlist8.m4s, ...., it didn't work. I don't know why, this supposed to be live video, the viewer that is watching the live from the beginning gets init.mp4, playlist0.m4s, playlist1.m4s, playlist2.m4s, ..... Someone that came after 5 minutes gets something like this init.mp4, playlist32.m4s, playlist33.m4s, playlist34.m4s, .... and so on, but so far it works only for the viewer that get's init.mp4, playlist0.m4s, playlist1.m4s, playlist2.m4s, ..... the video can't play for the others

ler
  • 1,466
  • 3
  • 20
  • 37
  • Is WebM required? If not you might try mp4/h.264. – heff Feb 02 '18 at 21:26
  • @heff WebM is not required, no problem if it's mp4 as long as it works well with Media Source Extensions API, if you have a working example i'll be appreciated if you put it as an answer – ler Feb 02 '18 at 23:07
  • Check my answer here https://stackoverflow.com/a/30643655/83169. You have to work on the client side so you can chunk the `fMP4` in `moof+mdat` segments. – Pablo Montilla Feb 05 '18 at 12:46
  • @PabloMontilla i saw your answer while i was searching but i didn't understand it, i'm a beginner to ffmpeg and mse, can you please post the code you mean as an answer, thanks – ler Feb 05 '18 at 22:08

1 Answers1

2

If you can use fMP4 instead of webm, you can generate a suitable fragmented MP4 using the following command line (geared towards low latency):

ffmpeg -i <input file> -c:v libx264 -profile:v main -level 3.2 -pix_fmt yuv420p -b:v <bitrate> -preset medium -tune zerolatency -flags +cgop+low_delay -movflags empty_moov+omit_tfhd_offset+frag_keyframe+default_base_moof+isml -c:a aac <output file.mp4>

Be sure to use the latest version.

You'll still need to parse the generated MP4 output to send the web client an initialization segment with the moov box, and pairs of moof+mdat boxes as segments for MSE to play video.

Pablo Montilla
  • 2,941
  • 1
  • 31
  • 35
  • thanks it needs lot of work, i found that i can use this command `ffmpeg -i -f hls -hls_segment_type fmp4 -c:v copy playlist.m3u8`, then send to the browser `init.mp4` followed by `playlist0.m4s, playlist1.m4s, playlist2.m4s, ...` this work fine , but the problem when i try it like this `init.mp4, playlist5.m4s, playlist6.m4s, ...` it didn't work, i don't know why. it suppose to be like live video, meaning someone can watch it from the beginning someone after 10 minutes, i mean you can't send `init.mp4, playlist0.m4s, playlist1.m4s, ...` to someone that came after 10 minutes – ler Feb 06 '18 at 14:40
  • You basically have two standard options: HLS (the only one supported by iOS devices) and DASH. Both can use fMP4, and both use a file that describes the different video segments (an .m3u8 or .xml manifest). If what you want is to allow people to see a video adapting to their bandwidth, then you need to use HLS or DASH and a suitable player on the client (Shaka [https://github.com/google/shaka-player] for example). If you want to stream a live event, you probably can adapt the HLS or DASH way to your needs. Otherwise you have to code yourself something doing what I'm telling you in the answer. – Pablo Montilla Feb 06 '18 at 14:50
  • what you suggested in your answer needs more code + the mp4 files are too big compared to `playlist.m4s`, if there a way i can play the video like this `init.mp4, playlist8.m4s, playlist9.m4s, ...` i mean not in successive way would be good, i will update my question so you understand what i mean – ler Feb 06 '18 at 15:12
  • You can simply change the `video.currentTime` to be the correct time. Your problem is that you are adding segments in the future, but the player tries to start at the beginning. You can see what is the time corresponding to the loaded segments by looking at `video.buffered`. – Pablo Montilla Feb 06 '18 at 15:18
  • i'm a beginner this is my first time with MSE and video, can you please show me how can i change `video.currentTime` to the correct one by looking at `video.buffered` like you said – ler Feb 06 '18 at 15:32
  • `video.currentTime = video.buffered.start(0);` – Pablo Montilla Feb 06 '18 at 15:34
  • it didn't work it gives me `Uncaught DOMException: Failed to execute 'start' on 'TimeRanges': The index provided (0) is greater than or equal to the maximum bound (0).`. when i send `init.mp4,1,2,3` it gives me the error above and when ever it receive a chunk it start the video from the beginning , when i send `init.mp4, 6,7,8` it gives me the error above and + it didn't play the video at all – ler Feb 06 '18 at 15:47
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/164632/discussion-between-pablo-montilla-and-ler). – Pablo Montilla Feb 06 '18 at 15:49