How to convert from stringstream to unsigned char vector and vice versa?

Question

I have std::stringstream ss;, containing binary data that I want to put into std::vector<unsigned char> my_vector;. Then, I want to take my_vector and use it to create a new std::stringstream new_ss identical to ss.

I have tried doing something similar to answers from this question:

Is there a more efficient way to set a std::vector from a stream?

and this question:

C++: vector to stringstream

ending up with:

std::copy(std::istream_iterator<unsigned char>(ss),
          std::istream_iterator<unsigned char>(),
          std::back_inserter(my_vector));

then:

std::copy(my_vector.begin(), 
          my_vector.end(),
          std::ostream_iterator<unsigned char>(new_ss));

but ss and new_ss are not the same. I suspected that the problem might be something to do with ignored whitespace so I tried making the vector in this way instead:

    std::string const& s = ss.str();
    my_vector.reserve(s.size());
    my_vector.assign(s.begin(), s.end());

but this did not fix the problem.

Any ideas on what could be wrong or advice on a better way to solve this problem would be greatly appreciated.

Did you use `std::ios::binary` to construct the string streams? — eerorika, Dec 15 '18 at 21:59
It would help to have a [mcve] and know in what way the strings differ... ("not the same" is not super precise) — Marc Glisse, Dec 15 '18 at 22:04
@ stringstreams come from cereal::BinaryInputArchive iarchive(ss) and are the result of serializing an object. — nellapizza, Dec 15 '18 at 22:20

Galik · Accepted Answer · 2018-12-16T02:20:04.600

I suspect this will be a fairly efficient method:

std::string const test_data = "hello world";

std::stringstream ss(test_data);

// discover size of data in stringstream
auto bof = ss.tellg();
ss.seekg(0, std::ios::end);
auto stream_size = std::size_t(ss.tellg() - bof);
ss.seekg(0, std::ios::beg);

// make your vector long enough
std::vector<unsigned char> v(stream_size);

// read directly in
ss.read((char*)v.data(), std::streamsize(v.size()));

// Now put it into a new stream object
std::stringstream new_ss;

// direct write again
new_ss.write((char const*)v.data(), std::streamsize(v.size()));

// check the result is the same as the original data
assert(new_ss.str() == test_data);

These could be wrapped up into functions to make them easier:

std::vector<unsigned char> to_vector(std::stringstream& ss)
{
    // discover size of data in stream
    ss.seekg(0, std::ios::beg);
    auto bof = ss.tellg();
    ss.seekg(0, std::ios::end);
    auto stream_size = std::size_t(ss.tellg() - bof);
    ss.seekg(0, std::ios::beg);

    // make your vector long enough
    std::vector<unsigned char> v(stream_size);

    // read directly in
    ss.read((char*)v.data(), std::streamsize(v.size()));

    return v;
}

std::stringstream to_stream(std::vector<unsigned char> const& v)
{
    std::stringstream ss;
    ss.write((char const*)v.data(), std::streamsize(v.size()));
    return ss;
}

int main()
{
    std::string const test_data = "hello world";

    std::stringstream ss(test_data);

    auto v = to_vector(ss);

    auto new_ss = to_stream(v);

    assert(new_ss.str() == test_data);
}

Maybe `reinterpret_cast(v.data())` is preferable w.r.t. `(char const*)v.data()`: makes it more clear we are using this "risky" feature https://stackoverflow.com/a/332086/2436175 — Antonio, Aug 06 '21 at 13:40

score 1 · Answer 2 · answered Dec 15 '18 at 22:10

Using std::istream_iterator<T> iterator or std::ostream_iterator<T> is bound to not work (in case if input) and just unnecessary slow (in case if output). The problem is that std::istream_iterator<T> skips leading whitespace and both iterators giing through the formatted interface which makes various checks which don’t really help.

You could use std::istreambuf_iterator<char> or std::ostreambuf_iterator<char>. I’d think that converting via the string interface is more effective, though.

score 1 · Answer 3 · answered Dec 15 '18 at 22:27

std::vector has an overload constructor that takes a pair of input iterators that define the sequence of contents of the new vector.

std::stringstream ss;

// Fill in your stringstream, then:

std::vector<unsigned char> my_vector{
    std::istreambuf_iterator<char>{ss},
    std::istreambuf_iterator<char>{}
};

How to convert from stringstream to unsigned char vector and vice versa?

3 Answers3