Splitting a byte[] into multiple byte[] arrays in C#

Question

I am trying to "chunk" up the bytes of an image. This will allow me to upload a large image in portions. I have the image currently stored as one large byte[]. I would like to split the byte array into byte[]'s with a maxlength of 512 elements. However, I'm not sure how to do this in the most efficient way.

Does anyone know how I can do this in the most efficient manner?

Hi chech this link here: http://stackoverflow.com/questions/20797/how-to-split-a-byte-array I hope it helps, bye — Mitja Bonca, Aug 05 '12 at 12:14
It seems fairly straightforward to come up with an O(N) solution (by iterating once through all of the bytes). Are you looking for something better than O(N)? — mbeckish, Aug 05 '12 at 12:15
possible duplicate of [Array slices in C#](http://stackoverflow.com/questions/406485/array-slices-in-c-sharp) — Shai, Aug 05 '12 at 12:18
Similar question http://stackoverflow.com/questions/11765336/how-to-divide-array-of-file-paths-into-several-smaller-arrays/11765423#11765423 — GSerjo, Aug 05 '12 at 12:19

score 16 · Answer 1 · edited May 23 '17 at 12:17

16

I wrote an extension for this, originally for strings, but decided to make it generic.

    public static T[] CopySlice<T>(this T[] source, int index, int length, bool padToLength = false)
    {
        int n = length;
        T[] slice = null;

        if (source.Length < index + length)
        {
            n = source.Length - index;
            if (padToLength)
            {
                slice = new T[length];
            }
        }

        if(slice == null) slice = new T[n];
        Array.Copy(source, index, slice, 0, n);
        return slice;
    }

    public static IEnumerable<T[]> Slices<T>(this T[] source, int count, bool padToLength = false)
    {
        for (var i = 0; i < source.Length; i += count)
            yield return source.CopySlice(i, count, padToLength);
    }

Basically, you can use it like so:

byte[] myBytes; // original byte array

foreach(byte[] copySlice in myBytes.Slices(10))
{
    // do something with each slice
}

Edit: I also provided an answer on SO using Buffer.BlockCopy here but BlockCopy will only work on byte[] arrays, so a generic version for strings wouldn't be possible.

edited May 23 '17 at 12:17

Community

1
1

answered Aug 05 '12 at 12:24

Chris Gessler

22,727
7
57
83

Not sure if the name is that well chosen. It's common that an array slice is referring to a part of the original array. (similar semantics to `ArraySegment`. – CodesInChaos Aug 05 '12 at 12:29
@CodesInChaos - I'm open to new names... Can you suggest one? – Chris Gessler Aug 05 '12 at 12:43
1

`CopySlice` would be pretty clear. Or `Subarray` analogous to `Substring`. Naming functions is so hard... – CodesInChaos Aug 05 '12 at 12:55
@CodesInChaos - Nice. Thanks. +1 for you :) – Chris Gessler Aug 05 '12 at 12:58
Wouldn't work for `byte[] myBytes = new byte[33]; foreach (byte[] copySlice in myBytes.Slices(16)) {....}` – L.B Aug 05 '12 at 13:18
@L.B - sorry... I went and grabbed my production version, it has added code to pad the final array, or not and works with odd numbers (left overs). – Chris Gessler Aug 05 '12 at 14:27

Marc Gravell · Accepted Answer · 2012-08-05T12:27:19.877

The most efficient method would be: not to. If you already have the image as a single byte[] then for local code, just specifying the offset and length (perhaps som ArraySegment-of-byte) is usually sufficient. If your upload API only takes byte[], then you still shouldn't chunk it completely; just use a single 512 buffer and use Buffer.BlockCopy to load it will successive pieces of the data. You may need to resize (Array.Resize) the final chunk, but at most 2 arrays should be needed.

Even better; avoid needing a byte[] in the first place: consider loading the data via a streaming API (this will work well if the data is coming from a file); just use Read (in a loop, processing the returned value etc) to populate chunks of max 512. For example (untested, just of illustration):

byte[] buffer = new byte[512];
while(true) {
    int space = 512, read, offset = 0;
    while(space > 0 && (read = stream.Read(buffer, offset, space)) > 0) {
        space -= read;
        offset += read;
    }
    // either a full buffer, or EOF
    if(space != 0) { // EOF - final
       if(offset != 0) { // something to send
         Array.Resize(red buffer, offset);
         Upload(buffer);
       }
       break;
    } else { // full buffer
       Upload(buffer);
    }
}

score 4 · Answer 3 · edited May 28 '14 at 12:00

4

public static IEnumerable<byte[]> Split(this byte[] value,int bufferLength){
   int countOfArray = value.Length / bufferLength;
   if(value.Length % bufferLength > 0)
      countOfArray ++;
   for(int i=0;i<countOfArray;i++)
   {
      yield return value.Skip(i * bufferLength).Take(bufferLength).ToArray();

   }
}

This is my extension what I used

edited May 28 '14 at 12:00

demongolem

9,474
36
90
105

answered May 28 '14 at 11:39

pelitahmet

41
5

score 2 · Answer 4 · answered May 04 '19 at 12:35

2

I know this is old but needed the same solution and following works perfectly for me hope this helps someone

private byte[][] ByteArrayToChunks(byte[] byteData, long BufferSize)
{
    byte[][] chunks = byteData.Select((value, index) => new { PairNum = Math.Floor(index / (double)BufferSize), value }).GroupBy(pair => pair.PairNum).Select(grp => grp.Select(g => g.value).ToArray()).ToArray();
    return chunks;
}

answered May 04 '19 at 12:35

Jack Gajanan

1,596
14
18

Looks really nice, but isn't really efficient. Nevertheless +1 for the alternative solution. – Stefan May 04 '19 at 12:39

score 1 · Answer 5 · answered Apr 21 '23 at 16:18

As of .NET 6, System.Linq has a built-in Chunk method for this.

For example:

using System.Linq;

...
  byte[] myBytes = new byte[131072];
  int myMaxChunkSizeInBytes = 512;
  foreach(byte[] chunk in myBytes.Chunk(myMaxChunkSizeInBytes))
  {
      // do something with each chunk
  }
...

MS Docs: System.Linq.Enumerable.Chunk(IEnumerable, Int32)

Splitting a byte[] into multiple byte[] arrays in C#

5 Answers5

Linked