How can I split an IEnumerable into groups of IEnumerable

Question

I have an IEnumerable<string> which I would like to split into groups of three so if my input had 6 items i would get a IEnumerable<IEnumerable<string>> returned with two items each of which would contain an IEnumerable<string> which my string contents in it.

I am looking for how to do this with Linq rather than a simple for loop

Thanks

Answer 1

This is a late reply to this thread, but here is a method that doesn't use any temporary storage:

public static class EnumerableExt
{
    public static IEnumerable<IEnumerable<T>> Partition<T>(this IEnumerable<T> input, int blockSize)
    {
        var enumerator = input.GetEnumerator();

        while (enumerator.MoveNext())
        {
            yield return nextPartition(enumerator, blockSize);
        }
    }

    private static IEnumerable<T> nextPartition<T>(IEnumerator<T> enumerator, int blockSize)
    {
        do
        {
            yield return enumerator.Current;
        }
        while (--blockSize > 0 && enumerator.MoveNext());
    }
}

And some test code:

class Program
{
    static void Main(string[] args)
    {
        var someNumbers = Enumerable.Range(0, 10000);

        foreach (var block in someNumbers.Partition(100))
        {
            Console.WriteLine("\nStart of block.");

            foreach (int number in block)
            {
                Console.Write(number);
                Console.Write(" ");
            }
        }

        Console.WriteLine("\nDone.");
        Console.ReadLine();
    }
}

However, do note the comments below for the limitations of this approach:

1. If you change the foreach in the test code to foreach (var block in someNumbers.Partition(100).ToArray()) then it doesn't work any more.

2. It isn't threadsafe.

Answer 2

var result = sequence.Select((s, i) => new { Value = s, Index = i })
                     .GroupBy(item => item.Index / 3, item => item.Value);

Note that this will return an IEnumerable<IGrouping<int,string>> which will be functionally similar to what you want. However, if you strictly need to type it as IEnumerable<IEnumerable<string>> (to pass to a method that expects it in C# 3.0 which doesn't support generics variance,) you should use Enumerable.Cast:

var result = sequence.Select((s, i) => new { Value = s, Index = i })
                     .GroupBy(item => item.Index / 3, item => item.Value)
                     .Cast<IEnumerable<string>>();

Answer 3

I know this has already been answered, but if you plan on taking slices of IEnumerables often, then I recommend making a generic extension method like this:

public static IEnumerable<IEnumerable<T>> Split<T>(this IEnumerable<T> source, int chunkSize)
{
    return source.Where((x,i) => i % chunkSize == 0).Select((x,i) => source.Skip(i * chunkSize).Take(chunkSize));
}

Then you can use sequence.Split(3) to get what you want.

(you can name it something else like 'slice', or 'chunk' if you don't like that 'split' has already been defined for strings. 'Split' is just what I happened to call mine.)

Answer 4

Inspired By @dicegiuy30's implementation, I wanted to create a version that only iterates over the source once and doesn't build the whole result set in memory to compensate. Best I've come up with is this:

public static IEnumerable<IEnumerable<T>> Split2<T>(this IEnumerable<T> source, int chunkSize) {
    var chunk = new List<T>(chunkSize);
    foreach(var x in source) {
        chunk.Add(x);
        if(chunk.Count <= chunkSize) {
            continue;
        }
        yield return chunk;
        chunk = new List<T>(chunkSize);
    }
    if(chunk.Any()) {
        yield return chunk;
    }
}

This way I build each chunk on demand. I wish I should avoid the List<T> as well and just stream that that as well, but haven't figured that out yet.

Answer 5

using Microsoft.Reactive you can do this pretty simply and you will iterate only one time through the source.

IEnumerable<string> source = new List<string>{"1", "2", "3", "4", "5", "6"};

IEnumerable<IEnumerable<string>> splited = source.ToObservable().Buffer(3).ToEnumerable();

Answer 6

We can improve @Afshari's solution to do true lazy evaluation. We use a GroupAdjacentBy method that yields groups of consecutive elements with the same key:

sequence
.Select((x, i) => new { Value = x, Index = i })
.GroupAdjacentBy(x=>x.Index/3)
.Select(g=>g.Select(x=>x.Value))

Because the groups are yielded one-by-one, this solution works efficiently with long or infinite sequences.

Answer 7

Mehrdad Afshari's answer is excellent. Here is the an extension method that encapsulates it:

using System.Collections.Generic;
using System.Linq;

public static class EnumerableExtensions
{
    public static IEnumerable<IEnumerable<T>> GroupsOf<T>(this IEnumerable<T> enumerable, int size)
    {
        return enumerable.Select((v, i) => new {v, i}).GroupBy(x => x.i/size, x => x.v);
    }
}

Answer 8

I came up with a different approach. It uses a while iterator alright but the results are cached in memory like a regular LINQ until needed.
Here's the code.

public IEnumerable<IEnumerable<T>> Paginate<T>(this IEnumerable<T> source, int pageSize)
{
    List<IEnumerable<T>> pages = new List<IEnumerable<T>>();
    int skipCount = 0;

    while (skipCount * pageSize < source.Count) {
        pages.Add(source.Skip(skipCount * pageSize).Take(pageSize));
        skipCount += 1;
    }

    return pages;
}