File.ReadAllLines or Stream Reader

Question

We can read file either by using StreamReader or by using File.ReadAllLines.

For example I want to load each line into a List or string[] for further manipulation on each line.

string[] lines = File.ReadAllLines(@"C:\\file.txt");

foreach(string line in lines)
{
     //DoSomething(line);
}

or

using (StreamReader reader = new StreamReader("file.txt"))
{
    string line;

    while ((line = reader.ReadLine()) != null)
    {
       //DoSomething(line); or //save line into List<string>
    }
}

//if list is created loop through list here

Application come across different size of text file. Which could grow from few KBs to MBs occasionally.

My question is that which one is preferred way and why one should be preferred over other?

http://stackoverflow.com/questions/8037070/whats-the-fastest-way-to-read-a-text-file-line-by-line — Yuval Itzchakov, Jun 02 '14 at 07:52
@Steve, speed and memory both. If larger size text file comes then application should efficiently read all lines. — Hassan, Jun 02 '14 at 07:59

Matthew Watson · Accepted Answer · 2014-06-02T08:11:16.143

40

If you want to process each line of a text file without loading the entire file into memory, the best approach is like this:

foreach (var line in File.ReadLines("Filename"))
{
    // ...process line.
}

This avoids loading the entire file, and uses an existing .Net function to do so.

However, if for some reason you need to store all the strings in an array, you're best off just using File.ReadAllLines() - but if you are only using foreach to access the data in the array, then use File.ReadLines().

edited Jun 02 '14 at 08:11

answered Jun 02 '14 at 08:05

Matthew Watson

104,400
10
158
276

2

@downvoter: Care to explain? Seems odd to downvote the correct answer... ;) – Matthew Watson Jun 02 '14 at 09:10
Probably downvoted because `File.ReadLines()` slurp the entire file into memory (a `string[]`). – Nicholas Carey Jul 27 '15 at 20:43
8

@NicholasCarey No, it's the version that DOESN'T do that: `public static IEnumerable ReadLines()`... hence my comment "without loading the entire file into memory" – Matthew Watson Jul 28 '15 at 07:50
1

@MatthewWatson I agree with you, though it's an old topic - upvoted for anybody going to read this and is beeing confused. – TripleEEE Nov 30 '16 at 14:31

score 25 · Answer 2 · answered Jun 02 '14 at 08:03

Microsoft uses a StreamReader in File.ReadAllLines:

    private static String[] InternalReadAllLines(String path, Encoding encoding)
    {
        Contract.Requires(path != null);
        Contract.Requires(encoding != null);
        Contract.Requires(path.Length != 0);

        String line;
        List<String> lines = new List<String>();

        using (StreamReader sr = new StreamReader(path, encoding))
            while ((line = sr.ReadLine()) != null)
                lines.Add(line);

        return lines.ToArray();
    }

score 5 · Answer 3 · edited Jul 26 '21 at 12:06

5

The StreamReader read the file line by line, it will consume less memory. Whereas, File.ReadAllLines read all lines at once and store it into string[], it will consume more memory. And if that string[] is larger than int.maxvalue then that will produce memory overflow(limit of 32bit OS).

So, for bigger files StreamReader will be more efficient.

edited Jul 26 '21 at 12:06

Yousha Aleayoub

4,532
4
53
64

answered Jun 02 '14 at 08:26

Nikunj.Patel

87
1
10

Given Sam's answer about ReadAllLines using StreamReader internally, is this answer invalid? – Stuart Dobson May 03 '23 at 00:12

File.ReadAllLines or Stream Reader

3 Answers3

Linked