Read/write a vector of Structs into a binary file and reading a vector of structs from a file in C++

Question

So i have a struct called info that contains different kind of variable and i created a vector inside of this struct. now i want to save this vector into a binary file so i can read it later. My code so far

    #include <iostream>
#include <fstream>
#include <vector>
#include <cstring>
struct MyStruct
{
    int a;
};
struct info
{
    char name[30];
    int age;
    char bbl[20];
    std::vector<MyStruct> my;
};
void create(std::vector<info>& test)
    {
        std::ofstream file("info.dat", std::ios::binary | std::ios::app); // output file stream

        // dump the contents of the vector in the file
        for (const info& inf : test)
            file.write(reinterpret_cast<const char*>(&inf), sizeof(inf));
    }

int main()
{
    info info1;
    // create a test vector
    std::vector<info> test;
    info1.age = 3443;
    std::cin >> info1.name;
    std::cin >> info1.bbl;
    MyStruct my;
    my.a = 4;
    info1.my.push_back(my);

    test.push_back(info1);
    
    std::cout << '\n';
    create(test);
    
    test.clear(); // clear the vector

    {
        std::ifstream file("info.dat", std::ios::binary); // input file stream

        // read the contents of the file into the vector
        info inf;
        while (file.read(reinterpret_cast<char*>(&inf), sizeof(inf)))
            test.push_back(inf);
    }

    // print out the contents of test
    for (int i = 0; i < test.size(); i++)
        std::cout << "{ " << test[i].name << ", " << test[i].age << ", " << test[i].bbl << " } ";
    std::cout << '\n';
}

But i got error on while (file.read(reinterpret_cast<char*>(&inf), sizeof(inf))) which is

Exception thrown at 0x7AD63E9E (vcruntime140d.dll) in ConsoleApplication9.exe: 0xC0000005: Access violation reading location 0xCC007364.

How can i fix that?

This will never work. You cannot read and write objects like this to a binary file as just raw bytes. Look up *object serialization*. — PaulMcKenzie, Dec 06 '21 at 22:01
You are going to need to learn how to serialize data. Writing the 24-or-so bytes occupied by a `std::vector` does not serialize the entire vector. For the same reason that writing a pointer doesn't serialize the data that the pointer points to. — Drew Dormann, Dec 06 '21 at 22:01
`vector` is commonly implemented as a set of pointers (one to the start of the data, one to the end of the data, and one to where the next value will go when added to the `vector`). The data is not ractually in the `vector`, so if you `write` a `vector` all you write is the pointers. And those pointers will be dangerous to handle later because they will probably be invalid. And if they are still valid you now have two different `vector`s referring to the same data and they both think the data is theirs and theirs alone to do with as they see fit. — user4581301, Dec 06 '21 at 22:07
For a simple structure like this you're probably better off writing your own `operator<<` and `operator>>` ([Notes on how to do that](https://stackoverflow.com/questions/4421706/what-are-the-basic-rules-and-idioms-for-operator-overloading)). Otherwise take a look at stuff like [protocol buffers](https://developers.google.com/protocol-buffers). — user4581301, Dec 06 '21 at 22:10
@Robert -- Just to show you that it cannot work, what is `sizeof(inf)`? It is a compile-time constant, maybe it's 60 bytes, whatever. Now imagine that vector has 1000 `MyStruct` instances in it. You are telling `read` and `write` to read/write 60 bytes of data, not 60 + 1000 * sizeof(MyStruct) bytes of data. So those calls could never work because simply, the last parameter is totally wrong. Also, even if the number of bytes were correct, simply look at the file yourself -- does what you see inside the file make sense? It wouldn't. — PaulMcKenzie, Dec 06 '21 at 22:14
Also, it seems that too many new C++ programmers take on this "binary file" reading/writing stuff, for no reason that I see. There must be hundreds, if not thousands of questions with the same issue, and almost all of them posted by new C++ programmers. So I wonder where the new programmers are getting this bad information, or being led into writing code like this. As mentioned before in the comments, I would suspect that learning how to overload `operator <<` and `>>` would be a priority for a new coder, and not obscure binary file/read writes. — PaulMcKenzie, Dec 06 '21 at 22:19
@PaulMcKenzie I think it's simply wishful thinking. New dev just wants to _write the vector_. `std::ofstream` has a `write` function, that can be passed the vector. New dev doesn't know what `write` is _actually doing_, because C++ would be so much harder if you had to know what every std function is _actually doing_. — Drew Dormann, Dec 06 '21 at 22:55
Binary I/O (e.g. using `file.write()`) does not work for any type that contains pointers, references, handles, or some other means of indirectly referring to data they manage. `std::vector` is a resizeable container, which means it's members are used to refer to other (dynamically allocated) data and the data is not (necessarily) part of the `std::vector` itself that will be output. That means NO `std::vector` can be simply output in a binary form. Nor can any struct/class type that (directly or indirectly) contains a `std::vector`. — Peter, Dec 07 '21 at 00:46

Andreas Wenzel · Accepted Answer · 2021-12-07T00:12:56.040

The std::vector object itself probably does not contain any actual data. Instead, it probably only contains bookkeeping information, for example

the number of valid elements,
the maximum number of elements for which memory has been allocated, and
a pointer to the start of the actual data.

Therefore, simply dumping the contents of the std::vector object to file will not be useful.

If you want to write the actual data of a std::vector to file, you must first decide how it should be stored in the file. One simple way of storing it would be to first write the number of elements as an int to the file, and then to write all of the individual elements to the file one after another. That way, when reading the file later, the reading function can easily find out how many elements it should read from the file, simply by reading the first value.

You may want to change your function create to the following:

void create(std::vector<info>& test)
{
    std::ofstream file("info.dat", std::ios::binary );

    for ( const info& inf : test )
    {
        //write "name" member to file
        file.write( reinterpret_cast<const char*>(&inf.name), sizeof inf.name );

        //write "age" member to file
        file.write( reinterpret_cast<const char*>(&inf.age), sizeof inf.age );

        //write "bbl" member to file
        file.write( reinterpret_cast<const char*>(&inf.bbl), sizeof inf.bbl );

        //write "my" member to file, giving it special treatment,
        //because it is a std::vector
        {
            int num_elements = inf.my.size();

            //write number of elements to file
            file.write( reinterpret_cast<const char*>(&num_elements), sizeof num_elements );

            //write the individual elements to file
            for ( int i = 0; i < num_elements; i++ )
            {
                file.write( reinterpret_cast<const char*>(&inf.my[i]), sizeof inf.my[i] );
            }
        }
    }

    //verify that stream is still in a good state
    if ( !file )
        throw std::runtime_error( "output error" );
}

Note that I removed std::ios::app in the code above, because it did not seem appropriate for what you are doing.

For reading the file contents, you can now use the following function:

void read(std::vector<info>& test)
{
    std::ifstream file("info.dat", std::ios::binary );

    info inf;

    //try reading new entries from file until end-of-file or error occurs
    for (;;)
    {
        //read "name" member from file
        file.read( reinterpret_cast<char*>(&inf.name), sizeof inf.name );

        //read "age" member from file
        file.read( reinterpret_cast<char*>(&inf.age), sizeof inf.age );

        //read "bbl" member from file
        file.read( reinterpret_cast<char*>(&inf.bbl), sizeof inf.bbl );

        //read "my" member from file, giving it special treatment,
        //because it is a std::vector
        {
            int num_elements;

            //read number of elements from file
            file.read( reinterpret_cast<char*>(&num_elements), sizeof num_elements );

            //don't start loop if loop counter is invalid
            if ( !file )
                break;

            //read the individual elements from file
            for ( int i = 0; i < num_elements; i++ )
            {
                MyStruct my;

                file.read( reinterpret_cast<char*>(&my), sizeof my );
                if ( file )
                    inf.my.push_back( my );
                else
                    break;
            }

            //stop main loop if data was not successfully read
            if ( !file )
                break;

            test.push_back( inf );
        }
    }
}

Your entire program will now look like this:

#include <iostream>
#include <fstream>
#include <vector>
#include <cstring>

struct MyStruct
{
    int a;
};

struct info
{
    char name[30];
    int age;
    char bbl[20];
    std::vector<MyStruct> my;
};

void create(std::vector<info>& test)
{
    std::ofstream file("info.dat", std::ios::binary );

    for ( const info& inf : test )
    {
        //write "name" member to file
        file.write( reinterpret_cast<const char*>(&inf.name), sizeof inf.name );

        //write "age" member to file
        file.write( reinterpret_cast<const char*>(&inf.age), sizeof inf.age );

        //write "bbl" member to file
        file.write( reinterpret_cast<const char*>(&inf.bbl), sizeof inf.bbl );

        //write "my" member to file, giving it special treatment,
        //because it is a std::vector
        {
            int num_elements = inf.my.size();

            //write number of elements to file
            file.write( reinterpret_cast<const char*>(&num_elements), sizeof num_elements );

            //write the individual elements to file
            for ( int i = 0; i < num_elements; i++ )
            {
                file.write( reinterpret_cast<const char*>(&inf.my[i]), sizeof inf.my[i] );
            }
        }
    }

    //verify that stream is still in a good state
    if ( !file )
        throw std::runtime_error( "output error" );
}

void read(std::vector<info>& test)
{
    std::ifstream file("info.dat", std::ios::binary );

    info inf;

    //try reading new entries from file until end-of-file or error occurs
    for (;;)
    {
        //read "name" member from file
        file.read( reinterpret_cast<char*>(&inf.name), sizeof inf.name );

        //read "age" member from file
        file.read( reinterpret_cast<char*>(&inf.age), sizeof inf.age );

        //read "bbl" member from file
        file.read( reinterpret_cast<char*>(&inf.bbl), sizeof inf.bbl );

        //read "my" member from file, giving it special treatment,
        //because it is a std::vector
        {
            int num_elements;

            //read number of elements from file
            file.read( reinterpret_cast<char*>(&num_elements), sizeof num_elements );

            //don't start loop if loop counter is invalid
            if ( !file )
                break;

            //read the individual elements from file
            for ( int i = 0; i < num_elements; i++ )
            {
                MyStruct my;

                file.read( reinterpret_cast<char*>(&my), sizeof my );
                if ( file )
                    inf.my.push_back( my );
                else
                    break;
            }

            //stop main loop if data was not successfully read
            if ( !file )
                break;

            test.push_back( inf );
        }
    }
}

int main()
{
    info info1;
    // create a test vector
    std::vector<info> test;
    info1.age = 3443;
    std::cin >> info1.name;
    std::cin >> info1.bbl;
    MyStruct my;
    my.a = 4;
    info1.my.push_back(my);

    test.push_back(info1);
    
    std::cout << '\n';
    create(test);
    
    test.clear(); // clear the vector

    read( test );

    // print out the contents of test
    for (int i = 0; i < test.size(); i++)
        std::cout << "{ " << test[i].name << ", " << test[i].age << ", " << test[i].bbl << " } ";
    std::cout << '\n';
}

This program has the following output:

TestName
TestBBL

{ TestName, 3443, TestBBL }

As you can see, the written data was read back properly.

@user4581301: I do not understand your comment. Please elaborate. — Andreas Wenzel, Dec 07 '21 at 20:16
[Mario Bros](https://en.wikipedia.org/wiki/Super_Mario_Bros.) Joke. Just like Princess Peach is not in the castle you just completed until the very end of the game, the `vector`'s data is not in the `vector`. — user4581301, Dec 07 '21 at 20:23

Barmak Shemirani · Answer 2 · 2021-12-07T00:45:38.327

You can read/write vector in binary, or vector of structure, as long as the struct is POD. The whole data can be obtained from vec.data() and the size of data is vec.size() * sizeof(vec[0])

In your example, the structure is not POD, this method can't be used. But the structure can be broken down in to POD + sub vector, and we can read/write each individual element.

Note however that this method is OS dependent. The binary data can be stored in big-endian or little-endian format, the structure size may vary between compilers. The file is not portable, at least not without additional work.

(This code requires C++ 17)

#include <iostream>
#include <fstream>
#include <vector>

struct info_pod
{ char name[30]; int age; char bbl[20]; };

struct info : info_pod
{ std::vector<int> subvec; };

int main()
{
    std::vector<info> src{ 
        { "name1", 100, "bbl1", { {10}, {11}, {12} } },
        { "name2", 101, "bbl2", { {10}, {11}, {12} } },
        { "name3", 102, "bbl3", { {10}, {11}, {12} } }
    };

    std::ofstream fout("info.dat", std::ios::binary); 
    if (!fout)
        return 0;

    for (auto& e : src)
    {
        fout.write((char*)&e, sizeof(info_pod));

        size_t count = e.subvec.size();
        fout.write((char*)&count, sizeof(count));

        auto vecsize = count * sizeof(e.subvec[0]);
        fout.write((char*)e.subvec.data(), vecsize);
    }

    fout.close();

    //read it back
    std::ifstream fin("info.dat", std::ios::binary);
    if (!fin)
        return 0;

    std::vector<info> dst;
    while (true)
    {
        info inf;
        fin.read((char*)&inf, sizeof(info_pod));
        if (fin.gcount() != sizeof(info_pod)) break;

        size_t count;
        fin.read((char*)&count, sizeof(count));
        if (fin.gcount() != sizeof(count)) break;

        inf.subvec.resize(count);
        auto vecsize = count * sizeof(inf.subvec[0]);
        fin.read((char*)inf.subvec.data(), vecsize);
        if (fin.gcount() != vecsize) break;

        dst.push_back(inf);
    }

    for (auto& e : dst)
    {
        std::cout << e.name << ',' << e.age << ',' << e.bbl << " MyStruct: ";
        for (auto& i : e.subvec) std::cout << i << ',';
        std::cout << '\n';
    }
}

Read/write a vector of Structs into a binary file and reading a vector of structs from a file in C++

2 Answers2