C++ copy N bytes from one stream to another

Question

Given ifstream as a source and ofstream as a destination what is the fastest way to copy N bytes starting from streampos(offset) in the source?

Edit

The easiest approach I've thought of was using a simple buffer:

char *buf = new char[N];
is.seekg(offset);
is.read(buf, N);
os.write(buf, N);
delete buf;

or smaller buffer of fixed size:

char buf[BUF_SIZE];
is.seekg(offset);
while (N > 0) {
    int n = min(BUF_SIZE, N);
    is.read(buf, n);
    os.write(buf, n);
    N -= n;
}

I'd like to know what can be faster or more efficient than this.

Please provide the solutions you have tried so far? And explain whey there have not worked. — Babra Cunningham, Jan 12 '17 at 10:42
The fastest way will depend on many details. you should try a few different ways and benchmark for the system you are interested in. — M.M, Jan 12 '17 at 11:05
@KamilKoczurek what will be the `InputIterator first` and `OutputIterator result` in this case? — Igor Myagkov, Jan 12 '17 at 11:06
Note that it should be `delete [] buf`, buf allocating a buffer on the stack is likely way faster... — Alexis Wilke, Dec 15 '21 at 22:57

score 3 · Answer 1 · edited May 23 '17 at 11:52

You could try different solutions. Most important thing about efficiency - measure.

You can look for sample implementations here: How to read a file into a vector elegantly and efficiently?

In most cases the bigger buffer the faster the reads/writes will go. Using iterator solutions to pass single bytes, for example like so:

std::copy(std::istream_iterator<char>(is),
          std::istream_iterator<char>(),
          std::ostream_iterator<char>(os));

although looks nice is pretty much the worst scenario in terms of efficiency - at least for the setup I've tested.

Reading the whole file at once from the specified offset to a large buffer gives best time results - you could try this unless you have some memory limits. To do this you should calculate the file size, read the data to a buffer (for example initialized like this: std::vector<char> buff(fileSize, 0)) and write it to output stream at one call.

Since you only want to copy N bytes then you should compare the value to the file size minus starting offset to still be able to make it in one big read/write.

For example:

// helper function
streampos getSizeToEnd(ifstream& is)
{
    auto currentPosition = is.tellg();
    is.seekg(0, is.end);
    auto length = is.tellg() - currentPosition;
    is.seekg(currentPosition, is.beg);
    return length;
}

int main()
{
    std::ifstream is;
    std::ofstream os;
    ...
    const auto offset = 100;
    is.seekg(offset);
    std::vector<char> buff(getSizeToEnd(is), 0);

    // read the data chunk to buffer and from buffer to output stream
    is.read(buff.data(), buff.size());
    os.write(buff.data(), buff.size());
    ...
}

C++ copy N bytes from one stream to another

1 Answers1