Obtain a Span over a struct without making a copy of the struct

Question

I have been experimenting with Span<T> as part of ReadOnlySequence<T> and System.IO.Pipelines.

I am currently trying to obtain a Span<T> over a struct without using unsafe code and without making a copy of that struct.

My struct is simply:

    [StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Unicode)]
    public struct Packet
    {
        public byte TestByte;
    }

Method 1 - which works - but feels "unsafe"

    //
    // Method 1 - uses Unsafe to get a span over the struct
    //
    var packet = new Packet();
    unsafe
    {
        var packetSpan = new Span<byte>(&packet, Marshal.SizeOf(packet));

        packetSpan[0] = 0xFF; // Set the test byte
        Debug.Assert(packet.TestByte == 0xFF, "Error, packetSpan did not update packet.");
            // ^^^ Succeeds
        packet.TestByte = 0xEE;
        Debug.Assert(packetSpan[0] == 0xEE, "Error, packet did not update packetSpan.");
            // ^^^ Succeeds
    }

Method 2 - which doesn't work as intended as it requires a copy

    //
    // Method 2
    //
    // This doesn't work as intended because the original packet is actually
    // coppied to packet2Array because it's a value type
    //
    // Coppies the packet to an Array of Packets
    // Gets a Span<Packet> of the Array of Packets
    // Casts the Span<Packet> as a Span<byte>
    //
    var packet2 = new Packet();

    // create an array and store a copy of packet2 in it
    Packet[] packet2Array = new Packet[1];
    packet2Array[0] = packet2;

    // Get a Span<Packet> of the packet2Array
    Span<Packet> packet2SpanPacket = MemoryExtensions.AsSpan<Packet>(packet2Array);

    // Cast the Span<Packet> as a Span<byte>
    Span<byte> packet2Span = MemoryMarshal.Cast<Packet, byte>(packet2SpanPacket);

    packet2Span[0] = 0xFF; // Set the test byte
    Debug.Assert(packet2.TestByte == 0xFF, "Error, packet2Span did not update packet2");
        // ^^^ fails because packet2 was coppied into the array, and thus packet2 has not changed.
    Debug.Assert(packet2Array[0].TestByte == 0xFF, "Error, packet2Span did not update packet2Array[i]");
        // ^^^ succeeds

    packet2.TestByte = 0xEE;
    Debug.Assert(packet2Span[0] == 0xEE, "Error, packet2 did not update in packet2Span");
        // ^^^ fails because packet2Span is covering packet2Array which has a copy of packet2 
    packet2Array[0].TestByte = 0xEE;
    Debug.Assert(packet2Span[0] == 0xEE, "Error, packet2 did not update in packet2Span");
        // ^^^ succeeds

Further research shows that

Span<T> can be implicitly cast from a byte[], eg, I could do

Span<byte> packetSpan = new Packet().ToByteArray();

But any current implementation of ToByteArray() I have is still making a copy of the Packet struct.

I can't do some something like:

Span<byte> packetSpan = (byte[])packet;
    // ^^ Won't compile

I think you can write user defined conversion operator(s). https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/operators/user-defined-conversion-operators — dropoutcoder, Jan 05 '20 at 02:06
I'm pretty sure that there's no way to do that for a struct in general without `unsafe`, because if you acquire a `Span` over all bytes of a struct you can potentially alter any bit in that struct in any way - that's inherently unsafe. — V0ldek, Jan 05 '20 at 02:07

score 4 · Accepted Answer · answered Jan 05 '20 at 02:33

There's no way of acquiring a Span<byte> over an arbitrary struct without unsafe, since such a span would allow you to change any bit of the struct in any way, possibly violating the type's invariants - that's inherently an unsafe operation.

Okay, but what about ReadOnlySpan<byte>? Notice that you had to put the StructLayoutAttribute on your struct for your code to be sensible. That should be a hint. Imagine trying to write an even simpler method, one that returns a byte[] for any arbitrary T where T : struct. You have to find out the size of the struct first, don't you? Well, how do you find out the size of a struct in C#? You can either use the sizeof operator, which requires an unsafe context and needs the struct to be an unmanaged type; or you can Marshall.SizeOf which is wonky and works only on structs with sequential or explicit byte layout. There's no safe, general way, thus you cannot do that.

The Span<T> and ReadOnlySpan<T> weren't designed with accessing struct bytes in mind, but rather with spanning fragments of arrays, which have a known size and are guaranteed to be sequential.

If you are confident that you know what you're doing, you can do that in an unsafe context - that's what it's for. But note that your solution with unsafe doesn't generalise to arbitrary structs, for the reasons noted above.

If you intend your struct to be used as a buffer for IO operations, you might want to look into fixed size buffers. They also require an unsafe context, but you can encapsulate the unsafeness inside your struct and return a Span<byte> to that fixed buffer. Basically anything that tackles the byte structure of objects in memory requires unsafe in .NET, as memory management is the thing that this "safety" refers to.

Both of these are such perfect answers. Thanks for putting the effort in I really appreciate that and the pointers to fixed size buffers. I need to choose one as the answer although I would like to choose both. So I am afraid this is a case of my RNG. — Rowan Smith, Jan 06 '20 at 05:13

Rook · Answer 2 · 2023-02-08T12:36:59.900

As it turns out, it is possible to do this, and in fact has been possible for quite some time... since dotnet core 2.1, if the docs are to be believed!

The magic ingredient is MemoryMarshal.CreateSpan, which "Creates a new span over a portion of a regular managed object."


        var packet = new Packet() { TestByte = 123 };
        var span = MemoryMarshal.CreateSpan<Packet>(ref packet, 1);
        var bytes = MemoryMarshal.Cast<Packet, byte>(span);
        bytes[0] = 100;
        
        Console.WriteLine(packet.TestByte);

Outputs 100 as you'd hope.

You're prevented from doing anything too stupid here, because MemoryMarshal.Cast raises an ArgumentException if the to- or from-types contain any managed references. MemoryMarshal.CreateSpan comes with a bunch of caveats and offers you a new way to shoot yourself in the foot, but hey... it doesn't require unsafe so go wild!

This method should be used with caution. It is dangerous because the length argument is not checked. Even though the ref is annotated as scoped, it will be stored into the returned span, and the lifetime of the returned span will not be validated for safety, even by span-aware languages.

(scoped ref is a dotnet 7/C# 11 feature, but the method is available in older versions of dotnet without that particular keyword)

The blurb does not mention pinning at all, and I don't believe you need to worry about it. The span is constructed around a managed reference to the underlying struct (the ref packet bit) which is not a dumb pointer. As such I'd expect relocations of the struct (eg. because the managed object it was a member of to was moved by the GC) would be handled transparently by the runtime.

Just to confirm, you can absolutely do stuff like this:

        var packet = new Packet() { TestByte = 123 };
        var span = MemoryMarshal.CreateSpan<Packet>(ref packet, 100);
        var bytes = MemoryMarshal.Cast<Packet, byte>(span);
        new Random().NextBytes(bytes);

and cause a crash. So whilst this isn't unsafe in the keyword sense, it is absolutely not safe.

The GC is not an issue if `CreateSpan` cannot be used with managed objects. — CookedCthulhu, Feb 08 '23 at 10:57
@Schmid the things you are creating a span from might be a member of a managed type, and as such its address in memory is subject to change when its owning instance is relocated by the GC. I think though that GC isn't an issue because the `ref packet` that the span is constructed with is a managed reference that will cope with GC relocations. I'll update my answer to suit. — Rook, Feb 08 '23 at 12:33

Obtain a Span over a struct without making a copy of the struct

2 Answers2