MediaSource error: This SourceBuffer has been removed from the parent media source

Question

I'm experimenting with the new MediaSource API available in Chrome.

I'm trying to append binary data on the fly from a WebSocket to the video media source.

Starting with the example at https://html5-demos.appspot.com/static/media-source.html, my code is currently:

var websocket = new WebSocket('ws://localhost:8080');
websocket.binaryType = 'arraybuffer';

var mediaSource = new MediaSource();
var buffer;
var queue = [];

var video = $('.video')[0];
video.src = window.URL.createObjectURL(mediaSource);

mediaSource.addEventListener('sourceopen', function(e) {
  video.play();

  buffer = mediaSource.addSourceBuffer('video/mp4; codecs="avc1.64001E"');

  buffer.addEventListener('updatestart', function(e) { console.log('updatestart: ' + mediaSource.readyState); });
  buffer.addEventListener('update', function(e) { console.log('update: ' + mediaSource.readyState); });
  buffer.addEventListener('updateend', function(e) { console.log('updateend: ' + mediaSource.readyState); });
  buffer.addEventListener('error', function(e) { console.log('error: ' + mediaSource.readyState); });
  buffer.addEventListener('abort', function(e) { console.log('abort: ' + mediaSource.readyState); });

  buffer.addEventListener('update', function() { // Note: Have tried 'updateend'
    if (queue.length > 0 && !buffer.updating) {
      buffer.appendBuffer(queue.shift());
    }
  });
}, false);

mediaSource.addEventListener('sourceopen', function(e) { console.log('sourceopen: ' + mediaSource.readyState); });
mediaSource.addEventListener('sourceended', function(e) { console.log('sourceended: ' + mediaSource.readyState); });
mediaSource.addEventListener('sourceclose', function(e) { console.log('sourceclose: ' + mediaSource.readyState); });
mediaSource.addEventListener('error', function(e) { console.log('error: ' + mediaSource.readyState); });

websocket.addEventListener('message', function(e) {
  if (typeof e.data !== 'string') {
    if (buffer.updating || queue.length > 0) {
      queue.push(e.data);
    } else {
      buffer.appendBuffer(e.data);
    }
  }
}, false);

I consistently get the error message: InvalidStateError: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source. after one append. It looks like the MediaSource is closing immediately after the call to buffer.appendData().

Any way to do this elegantly?

Note: chrome://media-internals/ doesn't return any useful information.

Answer 1

Ultimately the issue was that I was sending h264 video down the websocket. The MediaSource API only supports MPEG-DASH and VP8 with keyframed segments currently (on Chrome 35).

Additionally, once I tried VP8, I saw that I was adding some frames out of order.

• Adding if (buffer.updating || queue.length > 0) in websocket.onmessage was required.
• Adding if (queue.length > 0 && !buffer.updating) in buffer.addEventListener('update', ...) was also required.

Note: I applied the edits mentioned here to the code in the question, so the only issue with the code in the question is that the codec is wrong

Answer 2

Chrome is unbelievably picky when it comes to SourceBuffer codecs. Worse, it will return useless and misleading error messages as in case of the OP.

It looks like the MediaSource is closing immediately after the call to buffer.appendData()

That's exactly the case: Chrome is not happy with the video as it somehow doesn't exactly conform to its likings and simply closes the source buffer without notice.

Possible solutions to look for:

1. Transcode your MP4 files using ffmpeg, explained here (headline Fragmenting).
2. Use mp4file of mp4v2 to find out the exact MP4 codec, explained here and here.
3. If you don't have audio, omit the audio codec part: e.g. 'video/mp4; codecs="avc1.64001F"' instead of 'video/mp4; codecs="avc1.64001F, mp4a.40.2"'.

Answer 3

check it before

const mimeType = 'video/mp4; codecs="avc1.64001E"';
if (MediaSource.isTypeSupported(mimeType)) {
  console.info('Mimetype is', mimeType);
  // TODO ...
} else {
  console.error('Mimetype not supported', mimeType);
}