android AudioTrack playback short array (16bit)

Question

I have an application that playback audio. It takes encoded audio data over RTP and decode it to 16bit array. The decoded 16bit array is converted to 8 bit array (byte array) as this is required for some other functionality.

Even though audio playback is working it is breaking continuously and very hard to recognise audio output. If I listen carefully I can tell it is playing the correct audio.

I suspect this is due to the fact I convert 16 bit data stream into a byte array and use the write(byte[], int, int, AudioTrack.WRITE_NON_BLOCKING) of AudioTrack class for audio playback.

Therefore I converted the byte array back to a short array and used write(short[], int, int, AudioTrack.WRITE_NON_BLOCKING) method to see if it could resolve the problem.

However now there is no audio sound at all. In the debug output I can see the short array has data.

What could be the reason?

Here is the AUdioTrak initialization

            sampleRate      =AudioTrack.getNativeOutputSampleRate(AudioManager.STREAM_MUSIC);

            minimumBufferSize = AudioTrack.getMinBufferSize(sampleRate, AudioFormat.CHANNEL_OUT_STEREO, AudioFormat.ENCODING_PCM_16BIT);

            audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, sampleRate,
                    AudioFormat.CHANNEL_OUT_STEREO,
                    AudioFormat.ENCODING_PCM_16BIT,
                    minimumBufferSize,
                    AudioTrack.MODE_STREAM);

Here is the code converts short array to byte array

for (int i=0;i<internalBuffer.length;i++){

        bufferIndex = i*2;
        buffer[bufferIndex] = shortToByte(internalBuffer[i])[0];
        buffer[bufferIndex+1] = shortToByte(internalBuffer[i])[1];
    }

Here is the method that converts byte array to short array.

public short[] getShortAudioBuffer(byte[] b){
    short audioBuffer[] = null;
    int index = 0;
    int audioSize = 0;
    ByteBuffer byteBuffer = ByteBuffer.allocate(2);

    if ((b ==null) && (b.length<2)){
        return null;
    }else{
        audioSize = (b.length - (b.length%2));
        audioBuffer   = new short[audioSize/2];
    }

    if ((audioSize/2) < 2)
        return null;

    byteBuffer.order(ByteOrder.LITTLE_ENDIAN);


    for(int i=0;i<audioSize/2;i++){
        index = i*2;
        byteBuffer.put(b[index]);
        byteBuffer.put(b[index+1]);
        audioBuffer[i] = byteBuffer.getShort(0);
        byteBuffer.clear();
        System.out.print(Integer.toHexString(audioBuffer[i]) + " ");
    }
    System.out.println();

    return audioBuffer;
}

Audio is decoded using opus library and the configuration is as follows;

opus_decoder_ctl(dec,OPUS_SET_APPLICATION(OPUS_APPLICATION_AUDIO));
    opus_decoder_ctl(dec,OPUS_SET_SIGNAL(OPUS_SIGNAL_MUSIC));
    opus_decoder_ctl(dec,OPUS_SET_FORCE_CHANNELS(OPUS_AUTO));
    opus_decoder_ctl(dec,OPUS_SET_MAX_BANDWIDTH(OPUS_BANDWIDTH_FULLBAND));
    opus_decoder_ctl(dec,OPUS_SET_PACKET_LOSS_PERC(0));
    opus_decoder_ctl(dec,OPUS_SET_COMPLEXITY(10));     // highest complexity
    opus_decoder_ctl(dec,OPUS_SET_LSB_DEPTH(16)); // 16bit = two byte samples
    opus_decoder_ctl(dec,OPUS_SET_DTX(0));             // default - not using discontinuous transmission
    opus_decoder_ctl(dec,OPUS_SET_VBR(1));             // use variable bit rate
    opus_decoder_ctl(dec,OPUS_SET_VBR_CONSTRAINT(0));  // unconstrained
    opus_decoder_ctl(dec,OPUS_SET_INBAND_FEC(0));      // no forward error correction

Answer 1

Let's assume you have a short[] array which contains the 16-bit one channel data to be played. Then each sample is a value between -32768 and 32767 which represents the signal amplitude at the exact moment. And 0 value represents a middle point (no signal). This array can be passed to the audio track with ENCODING_PCM_16BIT format encoding.

But things are going weird when playing ENCODING_PCM_8BIT is used (See AudioFormat)

In this case each sample encoded by one byte. But each byte is unsigned. That means, it's value is between 0 and 255, while 128 represents the middle point.

Java has no unsigned byte format. Byte format is signed. I.e. values -128...-1 will represent actual values of 128...255. So you have to be careful when converting to the byte array, otherwise it will be a noise with barely recognizable source sound.

short[] input16 = ... // the source 16-bit audio data;

byte[] output8 = new byte[input16.length];

for (int i = 0 ; i < input16.length ; i++) {
  // To convert 16 bit signed sample to 8 bit unsigned
  // We add 128 (for rounding), then shift it right 8 positions
  // Then add 128 to be in range 0..255
  int sample = ((input16[i] + 128) >> 8) + 128;
  if (sample > 255) sample = 255; // strip out overload
  output8[i] = (byte)(sample); // cast to signed byte type
}

To perform backward conversion all should be the same: each single sample to be converted to exactly one sample of the output signal

byte[] input8 = // source 8-bit unsigned audio data;

short[] output16 = new short[input8.length];

for (int i = 0 ; i < input8.length ; i++) { 
  // to convert signed byte back to unsigned value just use bitwise AND with 0xFF
  // then we need subtract 128 offset
  // Then, just scale up the value by 256 to fit 16-bit range
  output16[i] = (short)(((input8[i] & 0xFF) - 128) * 256);
}

Answer 2

The issue of not being able to convert data from byte array to short array was resolved when used bitwise operators instead of using ByteArray. It could be due not setting the correct parameters in ByteArray or it is not suitable for such conversion.

Nevertheless implementing conversion using bitwise operators resolved the problem. Since the original question has been resolved by this approach, please consider this as the final answer.

I will raise a separate topic for playback issue.

Thank you for all your support.