Obtain a Span over a struct without making a copy of the struct

Question

I have been experimenting with Span<T> as part of ReadOnlySequence<T> and System.IO.Pipelines.

I am currently trying to obtain a Span<T> over a struct without using unsafe code and without making a copy of that struct.

My struct is simply:

    [StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Unicode)]
    public struct Packet
    {
        public byte TestByte;
    }

Method 1 - which works - but feels "unsafe"

    //
    // Method 1 - uses Unsafe to get a span over the struct
    //
    var packet = new Packet();
    unsafe
    {
        var packetSpan = new Span<byte>(&packet, Marshal.SizeOf(packet));

        packetSpan[0] = 0xFF; // Set the test byte
        Debug.Assert(packet.TestByte == 0xFF, "Error, packetSpan did not update packet.");
            // ^^^ Succeeds
        packet.TestByte = 0xEE;
        Debug.Assert(packetSpan[0] == 0xEE, "Error, packet did not update packetSpan.");
            // ^^^ Succeeds
    }

Method 2 - which doesn't work as intended as it requires a copy

    //
    // Method 2
    //
    // This doesn't work as intended because the original packet is actually
    // coppied to packet2Array because it's a value type
    //
    // Coppies the packet to an Array of Packets
    // Gets a Span<Packet> of the Array of Packets
    // Casts the Span<Packet> as a Span<byte>
    //
    var packet2 = new Packet();

    // create an array and store a copy of packet2 in it
    Packet[] packet2Array = new Packet[1];
    packet2Array[0] = packet2;

    // Get a Span<Packet> of the packet2Array
    Span<Packet> packet2SpanPacket = MemoryExtensions.AsSpan<Packet>(packet2Array);

    // Cast the Span<Packet> as a Span<byte>
    Span<byte> packet2Span = MemoryMarshal.Cast<Packet, byte>(packet2SpanPacket);

    packet2Span[0] = 0xFF; // Set the test byte
    Debug.Assert(packet2.TestByte == 0xFF, "Error, packet2Span did not update packet2");
        // ^^^ fails because packet2 was coppied into the array, and thus packet2 has not changed.
    Debug.Assert(packet2Array[0].TestByte == 0xFF, "Error, packet2Span did not update packet2Array[i]");
        // ^^^ succeeds

    packet2.TestByte = 0xEE;
    Debug.Assert(packet2Span[0] == 0xEE, "Error, packet2 did not update in packet2Span");
        // ^^^ fails because packet2Span is covering packet2Array which has a copy of packet2 
    packet2Array[0].TestByte = 0xEE;
    Debug.Assert(packet2Span[0] == 0xEE, "Error, packet2 did not update in packet2Span");
        // ^^^ succeeds

Further research shows that

Span<T> can be implicitly cast from a byte[], eg, I could do

Span<byte> packetSpan = new Packet().ToByteArray();

But any current implementation of ToByteArray() I have is still making a copy of the Packet struct.

I can't do some something like:

Span<byte> packetSpan = (byte[])packet;
    // ^^ Won't compile

I think you can write user defined conversion operator(s). https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/operators/user-defined-conversion-operators — dropoutcoder, Jan 05 '20 at 02:06
I'm pretty sure that there's no way to do that for a struct in general without `unsafe`, because if you acquire a `Span` over all bytes of a struct you can potentially alter any bit in that struct in any way - that's inherently unsafe. — V0ldek, Jan 05 '20 at 02:07

moien · Answer 1 · 2020-01-05T09:45:47.660

You have to do it in an unsafe context because it's unsafe by the real meaning of the word, because if you are not careful enough you will shoot yourself in the foot. Here's why:

Consider the following code:

Span<byte> GiveMeSpan() 
{
    MyLovelyStruct value = new MyLovelyStruct();
    unsafe 
    {
        return new Span<byte>(&value, sizeof(MyLovelyStruct));
    }
}

The instance of MyLovelyStruct which we create in GiveMeSpan() lives in the method's call stack and what your doing is taking its address, giving it to the Span<byte>, and returning the Span<byte>. Once a method returns it pops its stack frame, thus the memory that your MyLovelyStruct is living in will become free and may be reclaimed by the next method that the caller calls and corrupt it.

But that's not all, what if your MyLovelyStruct is living in an object field like this:

class MyLovelyClass 
{
    private MyLovelyStruct value;

    public void Foo() 
    {
        unsafe 
        {
            var span = new Span(&value, sizeof(MyLovelyStruct));
            Process(span);
        }
    }
}

// Declaration 
Process(Span<byte> span);

And a GC happens when Process() method is processing your MyLovelyStruct and MyLovelyClass suddenly gets moved in the memory (Yes, GC move's objects in the memory, read here)? Yes, your Span<byte> which points to MyLovelyStruct will no longer point to the new MyLovelyStruct address and your program becomes corrupt.

So in order to safely wrap a struct using Span<byte> or any other pointer type, you have to make sure:

The instance is living in a fixed memory position (for example in the stack or unmanaged memory like a memory block allocated by Marshal.AllocHGlobal)
The instance memory will not be claimed until you are done with the pointer

So unsafe keyword is needed and even if you can bypass it, it is up to you to warn the readers of your code about it.

Both of these are such perfect answers. Thanks for putting the effort in I really appreciate that and the clearer explanation/links to unsafe I need to choose one as the answer although I would like to choose both. So I am afraid this is a case of my RNG. — Rowan Smith, Jan 06 '20 at 05:13

score 4 · Accepted Answer · answered Jan 05 '20 at 02:33

There's no way of acquiring a Span<byte> over an arbitrary struct without unsafe, since such a span would allow you to change any bit of the struct in any way, possibly violating the type's invariants - that's inherently an unsafe operation.

Okay, but what about ReadOnlySpan<byte>? Notice that you had to put the StructLayoutAttribute on your struct for your code to be sensible. That should be a hint. Imagine trying to write an even simpler method, one that returns a byte[] for any arbitrary T where T : struct. You have to find out the size of the struct first, don't you? Well, how do you find out the size of a struct in C#? You can either use the sizeof operator, which requires an unsafe context and needs the struct to be an unmanaged type; or you can Marshall.SizeOf which is wonky and works only on structs with sequential or explicit byte layout. There's no safe, general way, thus you cannot do that.

The Span<T> and ReadOnlySpan<T> weren't designed with accessing struct bytes in mind, but rather with spanning fragments of arrays, which have a known size and are guaranteed to be sequential.

If you are confident that you know what you're doing, you can do that in an unsafe context - that's what it's for. But note that your solution with unsafe doesn't generalise to arbitrary structs, for the reasons noted above.

If you intend your struct to be used as a buffer for IO operations, you might want to look into fixed size buffers. They also require an unsafe context, but you can encapsulate the unsafeness inside your struct and return a Span<byte> to that fixed buffer. Basically anything that tackles the byte structure of objects in memory requires unsafe in .NET, as memory management is the thing that this "safety" refers to.

Both of these are such perfect answers. Thanks for putting the effort in I really appreciate that and the pointers to fixed size buffers. I need to choose one as the answer although I would like to choose both. So I am afraid this is a case of my RNG. — Rowan Smith, Jan 06 '20 at 05:13

Obtain a Span over a struct without making a copy of the struct

2 Answers2