42

I have a List<T> containing some data. I would like to pass it to a function which accepts ReadOnlySpan<T>.

List<T> items = GetListOfItems();
// ...
void Consume<T>(ReadOnlySpan<T> buffer)
// ...
Consume(items??);

In this particular instance T is byte but it doesn't really matter.

I know I can use .ToArray() on the List, and the construct a span, e.g.

Consume(new ReadOnlySpan<T>(items.ToArray()));

However this creates a (seemingly) unneccessary copy of the items. Is there any way to get a Span directly from a List? List<T> is implemented in terms of T[] behind the scenes, so in theory it's possible, but not as far as I can see in practice?

Selman Genç
  • 100,147
  • 13
  • 119
  • 184
Orion Edwards
  • 121,657
  • 64
  • 239
  • 328
  • I don't think you can until they implement this for `List`, in natural ways. Otherwise you can use reflection to access backing array and call `AsSpan` on it. – Selman Genç Sep 24 '18 at 10:11
  • 7
    If you're wanting to use `Span` for performance reasons, perhaps you shouldn't be starting with a `List` in the first place? – DavidG Sep 24 '18 at 10:15
  • 2
    The internal buffer could be reallocated by the next `Add` operation which means the `Span` will end up pointing to a dangling array and stale data. Imagine adding 2 items to an empty list, taking a span, then adding another item, causing a reallocation and then *modifying* all stored items. The span will end up looking at a stale copy of the data – Panagiotis Kanavos Sep 24 '18 at 10:16
  • 1
    I have deleted my reflection answer because i agree that it's not a good idea to use the internal array of the list. There are some issues with it: 1)if the list gets modified afterwards the array will be reallocated and the `ReadOnlySpan` holds a dangled array reference. 2) The internal array gets a size that will be doubled whenever it needs to be increased, so it might contain default values for that type which aren't contained in the list(`array.Length > list.Count`). If you are going to truncate that part you have to create a new array anyway. – Tim Schmelter Sep 24 '18 at 10:24
  • @PanagiotisKanavos thanks for the explanation. If you write that up as an answer I'll accept it for the question – Orion Edwards Sep 24 '18 at 10:53

4 Answers4

53

In .Net 5.0, you can use CollectionsMarshal.AsSpan() (source, GitHub issue) to get the underlying array of a List<T> as a Span<T>.

Keep in mind that this is still unsafe: if the List<T> reallocates the array, the Span<T> previously returned by CollectionsMarshal.AsSpan won't reflect any further changes to the List<T>. (Which is why the method is hidden in the System.Runtime.InteropServices.CollectionsMarshal class.)

svick
  • 236,525
  • 50
  • 385
  • 514
6

Thanks for all the comments explaining that there's no actual way to do it and how exposing the internal Array inside List could lead to bad behaviour and a broken span.

I ended up refactoring my code not to use a list and just produce spans in the first place.

void Consume<T>(ReadOnlySpan<T> buffer)
// ...

var buffer = new T[512]; 
int itemCount = ProduceListOfItems(buffer); // produce now writes into the buffer

Consume(new ReadOnlySpan<T>(buffer, 0, itemCount);

I'm chosing to make the explicit tradeoff of over-allocating the buffer once to avoid making an extra copy later on.

I can do this in my specific case because I know there will a maximum upper bound on the item count, and over-allocating slightly isn't a big deal, however there doesn't appear to be a generalisation here, nor would one ever get added as it would be dangerous.

As always, software performance is the art of making (hopefully favorable) trade-offs.

Orion Edwards
  • 121,657
  • 64
  • 239
  • 328
3

You can write your own CustomList<T> that exposes the underlying array. It is then on user code to use this class correctly.

In particular the CustomList<T> will not be aware of any Span<T> that you can obtain from the underlying backing array. After taking a Span<T> you should not make the list do anything to create a new array or create undefined data in the old array.

The C++ standard library allows user code to obtain direct pointers into vector<T> backing storage. They document the conditions under which this is safe. Resizing makes it unsafe for example.

.NET itself does something like this with MemoryStream. This class allows you access to the underlying buffer and indeed unsafe operations are possible.

usr
  • 168,620
  • 35
  • 240
  • 369
0

You can look at the example I did in Unity H4xx with IL + C# and made a plugin for Unity (.NET Framework 4.7.1 to 4.8.x and .NET Standard 2.1) that allows you to use CollectionMarshal.AsSpan<T> on a List<T> ... it's accomplished by simply breaking visibility / access modifier rules and making the internals of List<T> visible to steal the _items backing array from it and create a Span. From the Span, you can grab a pointer and literally read/write RAM, but it's not recommended to do so unless you know what you're doing.

Aaron Carter
  • 62
  • 6