chunking an enumerable via linq

Question

I have an IEnumerable. I want to chunk it up, ie say there are 500 elements, I want 10 50 element chunks out of it. I tried this

        while (true)
        {
            var page = ret.Take(pagesize).ToList();
            .. do stuff with page
        }

but that starts at the beginning every time. I am happy with either the above model (an external loop over each page). Or soemting that looks like this

 ret.ChunkIt(pagesize).Select(chunk => >do stuff with the page)

I would like to avoid something that materializes the whole list (it might be huge). Ie something that did

 List<List<object>> chunks = ret.Chunkit(100);

thats called the schlemeil the painter algorithm – pm100 Jun 08 '16 at 20:14 — pm100, Jun 08 '16 at 20:14
Not if collection implements IList then it is O(1) – Crowcoder Jun 08 '16 at 20:24 — Crowcoder, Jun 08 '16 at 20:24

konkked · Answer 1 · 2016-06-08T20:02:14.613

I've implemented something like this for an open source project I work on, made some changes to make it work as an extension

public static class Extension 
{

    private IEnumerable<T> Segment<T>( IEnumerator<T> iter, int size, out bool cont )
    {
        var ret= new List<T>( );
        cont = true;
        bool hit = false;
        for ( var i=0 ; i < size ; i++ )
        {
            if ( iter.MoveNext( ) )
            {
                hit = true;
                ret.Add( iter.Current );
            }
            else
            {
                cont = false;
                break;
            }
        }

        return hit ? ret : null;
    }

    /// <summary>
    /// Breaks the collection into smaller chunks
    /// </summary>
    public IEnumerable<IEnumerable<T>> Chunk<T>(this IEnumerable<T> collection, int size )
    {

        bool shouldContinue = collection!=null && collection.Any();

        using ( var iter = collection.GetEnumerator( ) )
        {
            while ( shouldContinue )
            {
                //iteration of the enumerable is done in segment
                var result = Segment( iter, size, out shouldContinue );

                if ( shouldContinue || result != null )
                    yield return result;

                else yield break;
            }
        }

    }
}

chunking an enumerable via linq

1 Answers1