IEnumerable.Take(0) on File.ReadLines seems not to dispose/close the File handle

Question

I have a function which Skips n lines of code and Takes y lines from a given file using File.ReadLines with Skip and Take combination. When I try to open the file given by filePath the next time:

string[] Lines = File.ReadLines(filePath).Skip(0).Take(0).ToArray();
using (StreamWriter streamWriter = new StreamWriter(filePath))
{
    // ...
}

I get a File in use by another process exception on the "using" line.

It looks like IEnumerable.Take(0) is the culprit, since it returns an empty IEnumerable without enumerating on the object returned by File.ReadLines(), which I believe is not disposing the file.

Am I right? Should they not enumerate to avoid this kind of errors? How to do this properly?

Answer 1

This is basically a bug in File.ReadLines, not Take. ReadLines returns an IEnumerable<T>, which should logically be lazy, but it eagerly opens the file. Unless you actually iterate over the return value, you have nothing to dispose.

It's also broken in terms of only iterating once. For example, you should be able to write:

var lines = File.ReadLines("text.txt");
var query = from line1 in lines
            from line2 in lines
            select line1 + line2;

... that should give a cross-product of lines in the file. It doesn't, due to the brokenness.

File.ReadLines should be implemented something like this:

public static IEnumerable<string> ReadLines(string filename)
{
    return ReadLines(() => File.OpenText(filename));
}

private static IEnumerable<string> ReadLines(Func<TextReader> readerProvider)
{
    using (var reader = readerProvider())
    {
        string line;
        while ((line = reader.ReadLine()) != null)
        {
            yield return line;
        }
    }
}

Unfortunately it's not :(

Options:

• Use the above instead of File.ReadLines

• Write your own implementation of Take which always starts iterating, e.g.

  public static IEnumerable<T> Take<T>(this IEnumerable<T> source, int count)
  {
      // TODO: Argument validation
      using (var iterator = source.GetEnumerator())
      {
          while (count > 0 && iterator.MoveNext())
          {
              count--;
              yield return iterator.Current;
          }
      }
  }
  

Answer 2

From the comment above File.ReadLines() in the Reference Source, it becomes obvious that the team responsible knew about this "bug":

Known issues which cannot be changed to remain compatible with 4.0:

• The underlying StreamReader is allocated upfront for the IEnumerable<T> before GetEnumerator has even been called. While this is good in that exceptions such as DirectoryNotFoundException and FileNotFoundException are thrown directly by File.ReadLines (which the user probably expects), it also means that the reader will be leaked if the user never actually foreach's over the enumerable (and hence calls Dispose on at least one IEnumerator<T> instance).

So they wanted File.ReadLines() to throw immediately when passed an invalid or unreadable path, as opposed to throwing when enumerating.

The alternative is simple: not calling Take(0), or instead not reading the file altogether if you aren't actually interested in its contents.

Answer 3

In my opinion, the root cause is Enumerable.Take iterator doesn't dispose an underlying iterator if the count is zero, since the code doesn't enter the foreach loop - see referencesource. If one modifies the code in following way the issue gets resolved:

static IEnumerable<TSource> TakeIterator<TSource>(IEnumerable<TSource> source, int count)
{
    foreach (TSource element in source)
    {
        if (--count < 0) break;
        yield return element;
    }
}