Buffered input versus standard input

Question

I was trying to read a long list of numbers (Around 10^7) from input file. Through some searching I found that reading the contents using buffer gives more performance when compared to reading the number one by one.

My second program is performing better than the first program. I am using a cin stream object in the first program and stringstream object in the second program. What is the difference between these two in terms of I/O performance?

#include <iostream>
using namespace std;

int main()
{
    int n,k;
    cin >> n >> k;
    int count = 0;
    while ( n-- > 0 )
    {
        int num;
        cin >> num;
        if( num % k == 0 )
            count++;
    }
    cout << count << endl;
    return 0;
}

This program is taking a longer time when compared to the following code using buffered input.

#include <iostream>
#include <sstream>
using namespace std;

int main()
{
    cin.seekg(0, cin.end);
    int length = cin.tellg();
    cin.seekg(0, cin.beg);
    char *buffer = new char[length];
    cin.read(buffer,length);
    stringstream ss(buffer);
    int n,k;
    ss >> n >> k;
    int result = 0;
    while( n-- )
    {
        int num;
        ss >> num;
        if( num % k == 0 )
            result++;
    }
    cout << result << endl;
    return 0;
}

You do know that `cin` *is* buffered? And also that `cin` (as a file) has an indeterminate size (so seeking to the end will not work as there is no real end). — Some programmer dude, Jul 24 '14 at 07:25
I was trying to reditect input from a file program.exe < input.txt. It worked fine for me — Ravi Chandra, Jul 24 '14 at 07:26
It doesn't really matter where the input is coming from, `cin` is not a file stream so you can't really change position. Once input has been read from `cin` you can't rewind to see the history, as well you can't fast forward into the future. Don't treat `cin` as a file, because it's not. `std::cin` can't be used for random access reading, even if you're getting the input from a redirected file. — Some programmer dude, Jul 24 '14 at 07:29
So what is the best way to read a big list of numbers from keyboard? — Ravi Chandra, Jul 24 '14 at 07:35
@JoachimPileborg `cin` is [a little special about buffering](http://stackoverflow.com/a/9371717/101999), actually. — Thanatos, Jul 24 '14 at 07:45
I would read into a [`std::vector`](http://en.cppreference.com/w/cpp/container/vector) using [`std::copy`](http://en.cppreference.com/w/cpp/algorithm/copy), [`std::istream_iterator`](http://en.cppreference.com/w/cpp/iterator/istream_iterator) and [`std::back_inserter`](http://en.cppreference.com/w/cpp/iterator/back_inserter). Redirecting from a file of one million random numbers (about 10 megs) it takes about 1.1 seconds on a recent PC to read it all. And that of course includes the time to read the numbers *as text* and converting them to `int`. — Some programmer dude, Jul 24 '14 at 07:52
@RaviChandra I can't replicate your claim that the second program is faster. On a 400 MiB input, the first takes ~0.1s to run 100 times, whereas the second takes ~17s. — Thanatos, Jul 24 '14 at 07:53
Oops, ignore that last comment. I wasn't properly structuring the input to the program. The second one is indeed faster. — Thanatos, Jul 24 '14 at 08:03

score 2 · Accepted Answer · edited May 23 '17 at 12:27

The second one will require ~twice the file's size in memory, otherwise, since it reads the entire file in one call, it will likely read data into memory as fast as the underlying storage can feed it, and then process it as fast as the CPU can do so.

It'd be good to avoid the memory cost, and in that respect, your first program is better. On my system, using an input called test.txt that looks like:

10000000 2
13
13
< 10000000-2 more "13"s. >

and your first program called a, and your second called b. I get:

% time ./a <test.txt 
0
./a < test.txt  1.70s user 0.01s system 99% cpu 1.709 total
% time ./b <test.txt
0
./b < test.txt  0.76s user 0.04s system 100% cpu 0.806 total

cin is not buffered by default, to keep "synchronized" with stdio. See this excellent answer for a good explanation. To make it buffered, I added cin.sync_with_stdio(false) to the top of your first program, and called the result c, which runs perhaps slightly faster:

% time ./c <test.txt
0
./c < test.txt  0.72s user 0.01s system 100% cpu 0.722 total

(Note: the times waffle around a bit, and I only ran a few tests, but c seems to be at least as fast as b.)

Your second program runs quickly because while not buffered, we can just issue one read call. The first program must issue a read call for each cin >>, whereas the third program can buffer (issue a read call every now and then).

Note that adding this line means you can't read from stdin using the C FILE * by that name, or call any library methods that would do so. In practice, this is likely to not be an issue.

Thanks. Your link helped me to understand the performance by using cin.sync_with_stdio(false); — Ravi Chandra, Jul 24 '14 at 09:56

Buffered input versus standard input

1 Answers1