How to stream MP3 data via WebSockets with node.js and socket.io?

Question

I have problems streaming MP3 data via WebSocket with node.js and socket.io. Everything seems to work but decodeAudioData doesn't play fair with me.

This is my toy server:

var app = require('http').createServer(handler)
  , io = require('socket.io').listen(app)
  , fs = require('fs')

app.listen(8081);

function handler (req, res) {
    res.writeHead(200, {
        'Content-Type': 'text/html',
    });
    res.end('Hello, world!');
}

io.configure('development', function() {
  io.set('log level', 1);

  io.set('transports', [ 'websocket' ]);
});

io.sockets.on('connection', function (socket) {
    console.log('connection established');

    var readStream = fs.createReadStream("test.mp3", 
                                         {'flags': 'r',
                                          'encoding': 'binary', 
                                          'mode': 0666, 
                                          'bufferSize': 64 * 1024});
    readStream.on('data', function(data) {
        console.log(typeof data);
        console.log('sending chunk of data')
        socket.send(data);
    });

    socket.on('disconnect', function () {
        console.log('connection droped');
    });
});

console.log('Server running at http://127.0.0.1:8081/');

The client receives the data as type string but I want to feed the data to decodeAudioData and it seems it doesn't like strings. The call to decodeAudioData results in the following error message:

Uncaught Error: SYNTAX_ERR: DOM Exception 12

I think decodeAudioData needs the data stored in an ArrayBuffer. Is there a way to convert the data?

This is the client code:

<script src="http://127.0.0.1:8081/socket.io/socket.io.js"></script>
<script>
    var audioBuffer = null;
    var context = null;
    window.addEventListener('load', init, false);
    function init() {
        try {
            context = new webkitAudioContext();
        } catch(e) {
            alert('Web Audio API is not supported in this browser');
        }
    }

    function decodeHandler(buffer) {
        console.log(data);
    }

    var socket = io.connect('http://127.0.0.1:8081');
    socket.on('message', function (data) {
            // HERE IS THE PROBLEM
        context.decodeAudioData(data, decodeHandler, function(e) { console.log(e); });
    });
</script>

I haven't found a solution using socket.io. See my own answer for a solution without socket.io. — Jan Deinhard, Mar 03 '12 at 18:35
Socket.io 1.0 have support for binary, I try it but doesnt work. Also I try with websocket, but with exactly the same error. You have an example of how you solve this?. — cmarrero01, Feb 28 '15 at 01:11
@cmarrero01 My solution at that time was using ws instead of Socket.io. But I know that it works with the current Socket.io version. On the server side I had to explicitly tell ws to send binary data like so `ws.send(array, { binary: true, mask: true });` Not sure if I set mask to true or false. HTH — Jan Deinhard, Feb 28 '15 at 05:04
Try removing `'encoding': 'binary',` from the `createReadStream` — Jan Swart, Oct 23 '16 at 15:45

Jan Deinhard · Accepted Answer · 2012-09-15T07:22:22.283

10

I've found a way to stream MP3 data via Websockets myself.

One problem was the chunk size of the MP3 data. It seems that the Web Audio API needs to be fed with valid MP3 chunks to be able to decode the data. Probably not surprising. In my demo app I provide a set of MP3 chunk files.

Also the quality of the audio is not perfect. I have some subtle glitches. I was able to improve that by sending larger chunks of MP3 data but there are still tiny crackles.

EDIT: I managed to improve the audio quality. It seems the Web Audio method decodeAudioData isn't really designed to decode continuos chunks of MP3 data.

edited Sep 15 '12 at 07:22

answered Jan 10 '12 at 08:12

Jan Deinhard

19,645
24
81
137

15

Can you please post your solution? I am also trying to stream MP3 data to websocket. – Light Jun 02 '13 at 14:15
1

Please supply your solution. – Brad.Smith Nov 15 '13 at 16:02
2

For anyone still looking, the solution is on his Github page: https://github.com/polaris/audio-demo – Casey Chow Jul 22 '14 at 17:31
3

@digitxp github project does not have anything related to streaming – lintu Nov 05 '16 at 07:36
How did you solve this? We're doing this but the audio sound is distorted/static noises. – Reyan Tropia Aug 25 '20 at 07:33
how did you improve the audio quality, i really want to know the solution – Chinh Nguyen Jul 05 '21 at 10:27

score 3 · Answer 2 · answered Jan 06 '12 at 19:01

3

In your case context.decodeAudioData expects an ArrayBuffer of the binary data, I would suggest converting your chunk to a base64 string, then to an ArrayBuffer client-side for the most predictable results. This script should be a good starting point for the client-side decode from base64 of the chunked data.

Adding a line with data = Base64Binary.decodeArrayBuffer(data); right after getting your data (base-64 encoded string) would do the trick...

answered Jan 06 '12 at 19:01

Tracker1

19,103
12
80
106

Thanks for your answer but I can't get it working with the script. The resulting ArrayBuffer is longer than it should be. I'd like to avoid Base64 anyway because of it's overhead. I think I've found a solution myself. I'm going to post it here soon. – Jan Deinhard Jan 10 '12 at 07:52
1

@Fair Dinkum Thinkum I would like to know your solution. – fyasar Dec 08 '12 at 22:12
@fyasar My solution was to use the `ws` module instead of socket.io. At that time socket.io wasn't capable of handling binary data. Not sure if that is still the case. – Jan Deinhard Dec 10 '12 at 15:14
@Fair Dinkum Thinkum can you please post your solution? – Light Jun 13 '13 at 04:43
@FairDinkumThinkum Socket.io 1.0 have support for binary data, but I stil cant make it works.. http://socket.io/blog/introducing-socket-io-1-0/#binary you can share your solution ? – cmarrero01 Feb 28 '15 at 01:27
@cmarrero01 I'd love to share it but I can't. I lost the code. I know that it works with Socket.io now because a colleague implemented the solution with Socket.io to support older browsers some time ago. I remember that is was tricky because of sparse documentation. – Jan Deinhard Feb 28 '15 at 04:59

score 0 · Answer 3 · edited Sep 18 '12 at 09:35

0

It seems that socket.io still doesn't support Binary transfer. So websocket.io can be used here.

edited Sep 18 '12 at 09:35

j0k

22,600
28
79
90

answered Jun 27 '12 at 04:48

Anant Gupta

488
1
8
13

How to stream MP3 data via WebSockets with node.js and socket.io?

3 Answers3

Linked