conversion from .csv file of large dimension to a 2D array

Question

i have HOG features stored in .csv file, which is of dimension 1967 X 2916. when i tried to store the value in a simple 2D array i am not able to carry out the process. but can run till a dimension of 88 X 2916. can you tell me the reason behind this? the features i use stored in .csv format can be found here https://www.dropbox.com/s/j04echzhwjwhgjk/data.csv?dl=0

the code looks something like this.

 #include<fstream>
 #include <sstream>

 using namespace std;

 int main(){

 float dataset[1967][2916];
 ifstream file("data.csv");

for(int row = 0; row < 1967; ++row)
{
    std::string line;
    std::getline(file, line);
    if ( !file.good() )
        break;

    std::stringstream iss(line);

    for (int col = 0; col < 2916; ++col)
    {
        std::string val;
        std::getline(iss, val, ',');
        if ( !iss.good() )
            break;

        std::stringstream convertor(val);
        convertor >> dataset[row][col];
    }
}

for(int p=0;p<1967;++p)
{
    cout<<endl;
    for(int q=0;q<2916;++q)
    {
        cout<<dataset[p][q]<<"\t";
    }
}

return 0;

}

Answer 1

This is likely a stack overflow, which is ironic, considering where you posted this. The stack is often limited to about 1-2 MB (this can be changed programatically).

You do a quick test by making the dataset global, which moves it off the stack:

float dataset[1967][2916];
int main(){
...

If the test works, you should put dataset on the heap.

Related: C/C++ maximum stack size of program