Efficient way of copying an array of nullables to array of non-nullables of the same base type?

Question

Is there an efficient way of copying an array of nullables (say byte?[]) to an array of non-nullables (say byte[]) assuming that the source array is guaranteed to not have any nullables (and if it does, it's ok to throw an exception)? Obviously, I can loop over the indices and copy each element individually.

This does not work. It compiles, but throws an ArrayTypeMismatchException at run-time.

 byte?[] sourceNullable = new byte?[]{1,2,3};
 byte[] destNonNullable = new byte[3];

 Array.Copy(sourceNullable,destNonNullable,3);

This will work but I am looking for something "better"

for(int i=0;i<3;i++) {
    destNonNullable[i] = sourceNullable[i] ?? 0;
}

I'm willing to accept the answer: what's wrong with the explicit loop? And why are you wasting time optimizing this? :)

Edit: I tried using the Linq style Cast<>(), but that turns out to be much slower. The time summary from my code below:

for loop = 585 ms

Linq Cast = 3626 ms

The input image file is a sparse array, filled with sections of nulls.

        uint rowsize = 16;
        Stopwatch sw = new Stopwatch();
        sw.Start();
        for (UInt32 address = start & 0xFFFFFFF0; address <= last; address += rowsize)
        {
            Int32 imageOffset = (Int32)(address - start);
            Int32 maxRowLen = (int)rowsize;
            if (maxRowLen + imageOffset > image.Length) maxRowLen = (image.Length - imageOffset);

            if (maxRowLen == 0) throw new Exception("this should not happen");

            int ptr = 0;
            while (ptr < maxRowLen)
            {
                while (ptr < maxRowLen && image[imageOffset + ptr] == null) ptr++;
                int startOffset = ptr;
                while (ptr < maxRowLen && image[imageOffset + ptr] != null) ptr++;
                int stopOffset = ptr;

                if (startOffset < maxRowLen)
                {
 #if false
                    int n = stopOffset - startOffset;
                    byte[] subdata = new byte[n];
                    for (int i = 0; i < n; i++)
                    {
                        subdata[i] = image[imageOffset + startOffset + i] ?? 0;
                    }
 #else
                    byte[] subdata = image.Skip(imageOffset + startOffset).Take(stopOffset - startOffset).Cast<byte>().ToArray();
 #endif
                    IntelHexRecord rec = new IntelHexRecord((uint)(address + startOffset), subdata);
                    records.Add(rec);
                }
            }
        }
        sw.Stop();
        Console.WriteLine("elapsed: {0} ms", sw.ElapsedMilliseconds);

I doubt that you'll find anything more efficient than copying each value individually. After all, each nullable value is a different size to the non-nullable, so you can't just blit the bytes. — Jon Skeet, Jul 26 '13 at 20:11
what's wrong with the explicit loop? And why are you wasting time optimizing this? :) — ToastyMallows, Jul 26 '13 at 20:12
I assume Array.Copy from `byte[]` to `byte[]` is more efficient that an explicit loop, so I was looking for something similar for `byte?[]` to `byte[]`. — Mark Lakata, Jul 26 '13 at 20:14
Remember, a nullable byte is nothing more than a byte plus a bool indicating whether the byte is nullable or not. You can't fit 257 possible values in 8 bits, so there's got to be an extra bit somewhere, and its in the bool. Therefore a byte array is half the size of a byte? array with the same number of elements. There's no magic way to quickly copy half the bytes. If the loop is too slow because of range checks on the arrays, you can use unsafe code to freeze the arrays in place, make pointers to them, and copy them that way. — Eric Lippert, Jul 26 '13 at 21:10
What is the runtime, in your sample, for a normal array copy with normal, non-nullable types? That should give you the theoretical upper margin... And I think byte is a bad example, nobody will make a byte? Array and then worry about performance... — , Jul 26 '13 at 21:12
I am manipulating an Intel "hex" file that represents flash memory. The flash memory is byte addressed and any single byte can not be programmed. I am using `null` to represent an unprogrammed byte and `0x00 to 0xFF` represent a programmed byte. Is this not a reasonable use of `byte?[]` The larger application is merging several hex files together. — Mark Lakata, Jul 26 '13 at 23:14

Sergey Kalinichenko · Answer 1 · 2013-07-26T20:56:20.147

You can use LINQ, like this:

byte[] destNonNullable = sourceNullable.Cast<byte>().ToArray();

However, this is not faster than what you are doing. If you need a faster way of copying a fixed number of bytes known at compile time, you can eliminate the loop overhead, which is not that big, but should work if you must squeeze the last CPU cycle out of this:

byte[] destNonNullable = new[] {
    sourceNullable[0].Value
,   sourceNullable[1].Value
,   sourceNullable[2].Value
};

You can also reduce the overhead by unwinding the loop. For example, if you know that the number of bytes that you would like to copy is divisible by four, you can do this:

Debug.Assert(N % 4 == 0); // Otherwise, the loop below wouldn't stop
for (int i = 0 ; i != N ; i += 4) {
    destNonNullable[i] = sourceNullable[i].Value;
    destNonNullable[i+1] = sourceNullable[i+1].Value;
    destNonNullable[i+2] = sourceNullable[i+2].Value;
    destNonNullable[i+3] = sourceNullable[i+3].Value;
}

I upvoted at first, thinking that this would be faster. It's not. It's MUCH slower. — Mark Lakata, Jul 26 '13 at 20:46
+1 to force me to learn that loop unrolling is still relevant in C#. http://stackoverflow.com/questions/2349211/when-if-ever-is-loop-unrolling-still-useful and http://jeff.xanga.com/583199309/loop-performance-in-c/ — Mark Lakata, Jul 26 '13 at 23:23

Efficient way of copying an array of nullables to array of non-nullables of the same base type?

1 Answers1