How do I efficiently get a subset of a byte array (first N elements) in C#?

Question

I have an byte array buffer of max size 1K. I want to write out a subset of the array (the start of the subset will always be element 0, but the length we're interested in is in a variable).

The application here is compression. I pass in a buffer to a compression function. For simplicity, assume the compression will lead to data that is equal, or less than 1K bytes.

byte[] buffer = new byte[1024];
while (true)
{
    uncompressedData = GetNextUncompressedBlock();
    int compressedLength = compress(buffer, uncompressedData);

    // Here, compressedBuffer[0..compressedLength - 1] is what we're interested in

    // There's a method now with signature Write(byte[] compressedData) that
    // I won't be able to change. Short of allocating a custom sized buffer,
    // and copying data into the custom sized buffer... is there any other
    // technique I could use to only expose the data I want?
}

I'd really like to avoid a copy here -- it seems completely unnecessary as all of the data needed is in buffer already.

Chris Gessler · Answer 1 · 2012-05-10T18:17:39.737

Buffer.BlockCopy would be my choice.

Microsoft example: http://msdn.microsoft.com/en-us/library/system.buffer.blockcopy.aspx

const int INT_SIZE = 4;
int[] arr = { 2, 4, 6, 8, 10, 12, 14, 16, 18, 20 };
Buffer.BlockCopy(arr, 3 * INT_SIZE, arr, 0 * INT_SIZE, 4 * INT_SIZE);
foreach (int value in arr)
   Console.Write("{0}  ", value);
// The example displays the following output:
//       8  10  12  14  10  12  14  16  18  20

Your code would look like:

uncompressedData = GetNextUncompressedBlock();      
int compressedLength = compress(buffer, uncompressedData);
Buffer.BlockCopy(buffer, 0, buffer, 0, compressedLength);

score 7 · Answer 2 · answered May 10 '12 at 18:23

7

    byte[] b = new Byte[] {1, 2, 3, 4, 5};
    IEnumerable<Byte> middle = b.Skip(2).Take(3);

This should let you get any middle section that you like. This most likely makes a copy, but I don't think you should attempt to avoid that.

answered May 10 '12 at 18:23

CodeHxr

855
8
17

score 6 · Accepted Answer · answered May 10 '12 at 18:01

6

If you can't change the method signature, then you're stuck. You can't create a "view" over a byte array with type byte[]. The ideal solution for that would be for the operation to either take an ArraySegment<byte> or a byte[] followed by an offset and a count. If you really can't change the Write method, then unfortunately you're stuck with creating a new array and copying the data to it.

answered May 10 '12 at 18:01

carlosfigueira

85,035
14
131
171

Wow, you've helped me out with WCF before (as a different user). Thanks again for the tip Carlos +1. – jglouie May 10 '12 at 18:03

score 3 · Answer 4 · answered May 10 '12 at 18:08

There's no way you can do it. Array.Resize instead of changing the length of the array, simply copies it to a new array instance.

However, you can use the Buffer class, which has better performance:

Buffer provides methods to copy bytes from one array of primitive types to another array of primitive types, get a byte from an array, set a byte in an array, and obtain the length of an array. This class provides better performance for manipulating primitive types than similar methods in the System.Array class.

This match your needs, because you have an array of bytes, and byte is a primitive type.

score 2 · Answer 5 · answered May 10 '12 at 18:02

If method signature is (byte[]) you can't do anything but copy.

If you can change signature:

streams support writing subsets of arrays, so it is not unusual request to have signature like Stream.Write:

public abstract void Write( byte[] buffer, int offset, int count)
the other option is to pass IEnumerable<byte> so you can slice your array any way you want without copy.

How do I efficiently get a subset of a byte array (first N elements) in C#?

5 Answers5

Linked