NodeJS Convert Int16Array binary Buffer to LINEAR16 encoded raw stream for Google Speech API

Question

I'm trying to convert speech to text in node server where speech recording happens in the browser using AudioContext. I'm Able to send int16Array buffer(recorded data) to my node server through a WebSocket connection of binaryType:arraybuffer.

this.processor.onaudioprocess = (e) => {
    // this.processAudio(e)
    for (
        var float32Array = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize),
        len = float32Array.length,
        int16Array = new Int16Array(len);
        len--;)
        int16Array[len] = 32767 * Math.min(1, float32Array[len]);
    this.socket.send(int16Array.buffer);
};

In server, data is received as

<Buffer 66 6f 6f ...>

Now I would like to parse or convert to a readable stream so that I can pipe to Google speech recognizeStream.

  function processAudioBuffer(int16ArrayBuffer) {
    console.log("Received stream :", int16ArrayBuffer, typeof 
    recognizeStreams[userId]);
    const recognizer = getGoogleSpeechStreamRecognizer();

    if (recognizer) {

    /* HERE I NEED SOMETHING WHICH MAKES MY BUFFER COMPATIBLE WITH GOOGLE SPEECH API */

    // tried with streamifier but no luck
    // streamifier.createReadStream(int16ArrayBuffer).pipe(recognizer);

    // also tried with Record which is used in google-cloud-node-samples to record stream from connected mic device, but no luck
    var file = new Record({
       path: `${userId}.raw`,
       encoding: 'arraybuffer',
       contents: int16ArrayBuffer
    });
    file.pipe(recognizer);

    } else {
         console.log('user stream is not yet created');
    }
 }

recognizer throws following error:

Error: write after end
at writeAfterEnd (/Users/demo/node_modules/duplexify/node_modules/readable-stream/lib/_stream_writable.js:222:12)
at Writable.write (/Users/demo/node_modules/duplexify/node_modules/readable-stream/lib/_stream_writable.js:262:20)
at Duplexify.end (/Users/demo/node_modules/duplexify/index.js:223:18)
at Record.pipe (/Users/demo/node_modules/record/index.js:70:14)
at processAudioBuffer (/Users/demo/app.js:87:10)
at WebSocket.incoming (/Users/demo/app.js:104:7)
at emitTwo (events.js:106:13)
at WebSocket.emit (events.js:191:7)
at Receiver._receiver.onmessage (/Users/demo/node_modules/ws/lib/WebSocket.js:146:54)
at Receiver.dataMessage (/Users/demo/node_modules/ws/lib/Receiver.js:380:14)

Some ideas here: http://stackoverflow.com/questions/16038705/how-to-wrap-a-buffer-as-a-stream2-readable-stream — Michael Tallino, Apr 19 '17 at 20:37
Thanks, @MichaelTallino I tried it with streamifier & stream but no luck. — Siva Kumar, Apr 20 '17 at 16:15
you can change your int for loop to: `const int16Array = Int16Array.from(float32Array, (k) => 32767 * Math.min(1, k))` and that will do the transformation in a much prettier way — Samuel Thompson, Jan 02 '20 at 18:30

score 3 · Answer 1 · answered Apr 20 '17 at 17:12

3

Solved it !!! We can write the buffer directly to recognizerStream which created from GoogleSpeech as follows:

const recognizer = getGoogleSpeechStreamRecognizer();

recognizer.write(int16ArrayBuffer)

answered Apr 20 '17 at 17:12

Siva Kumar

715
1
7
21

I WISH I COULD HUG YOU!!!!!! I spent 6 hours trying to figure out how to do exactly what you just did! – Samuel Thompson Jan 02 '20 at 17:53
did you get this working to return results from the api? if so how did you get them? I tried also adding an `.on('data'.... ` to `recognizer` (on a new line after the write) but get various errors. – bionara Dec 02 '20 at 11:27
Really sorry to say that, it is been 3 years and I forgot what I did at that time to make it work – Siva Kumar Dec 02 '20 at 16:57

NodeJS Convert Int16Array binary Buffer to LINEAR16 encoded raw stream for Google Speech API

1 Answers1

Linked