How to prevent stackoverflow error when working with large array? C

Question

I have code to sort 60000 lines into arrays for each data type

It worked for 1 line but when looping through many lines I get stack overflow error.

This is how I defined structs.

int main(void) {

typedef struct values{
    struct values* Time;
    struct values* RESP;
    struct values* PLETH;
    struct values* II;
    struct values* V;
    struct values* AVR;
    struct values* ABP;     
}values;

    values array[60000];

Here is a sample of the data file

Here is the code I used to read the file.

char line[80];
FILE* pointer = fopen("bidmc_04_Signals.csv", "r");
fgets(line, 60000, pointer);//moves pointer past 1st line
printf(line); 
fgets(line, 60000, pointer);
printf(line);

Here is the array generation part of my code

    int i = 0;
    int c = 0;
    int row_count = 0;
    int row_limit = 60000;
        while (row_count < row_limit) {
            fgets(line, 60, pointer);
            char*p = strtok(line, ",");
            c = 0;
            while (p != NULL)
            {               
                if (c == 0)
                    strcpy(array[i].Time, p);
                    array[i].Time = malloc(10000);

                if (c == 1)
                    strcpy(array[i].RESP, p);
                    array[i].RESP = malloc(10000);

                if (c == 2)
                    strcpy(array[i].PLETH, p);
                    array[i].PLETH = malloc(10000);

                if (c == 3)
                    strcpy(array[i].II, p);
                    array[i].II = malloc(10000);

                if (c == 4)
                    strcpy(array[i].V, p);
                    array[i].V = malloc(10000);

                if (c == 5)
                    strcpy(array[i].AVR, p);
                    array[i].AVR = malloc(10000);

                if (c == 6)
                    strcpy(array[i].ABP, p);
                    array[i].ABP = malloc(10000);

                p = strtok(NULL, ",");
                c++;
                }

            row_count++;
            i++;        
        }
        
    fclose(pointer);
    for (i = 0; i < 60000; i++) {
    
    free(array[i].RESP);
    free(array[i].PLETH);
    free(array[i].II);
    free(array[i].V);
    free(array[i].AVR);
    free(array[i].ABP);
}

}

As you can see I tried using malloc() to move array data to heap but it didn't work. How do I prevent the code from getting a stack overflow error?

Perhaps you need to explain what `array` is, because as it stands, there is no large array allocated on the stack here. — goodvibration, Dec 20 '20 at 07:59
Note that a thread has a limited stack size of a few MB (see my answer here: https://stackoverflow.com/a/64085509/4561887). So, for really large objects, you must dynamically allocate onto the heap. — Gabriel Staples, Dec 20 '20 at 08:00
When asking for help with code, please post a [Minimal Reproducible Example](https://stackoverflow.com/help/minimal-reproducible-example) so that we can see exactly what is going on and so we can try it ourselves. From what we _can_ see, you certainly have memory errors (`malloc(10000)` when `sizeof values` is probably 56 :( ) which can literally result in anything happening. — Ken Wayne VanderLinde, Dec 20 '20 at 08:11
Note that `strcpy(array[i].Time, p); array[i].Time = malloc(10000);` are **not** part of the `if (c == 0)`. Only `strcpy(array[i].Time, p);` is part of the `if()` block. — chux - Reinstate Monica, Dec 20 '20 at 08:22
Aside: `printf(line);` is considered dangerous, and opens the code to malicious exploits – the data could contain format specifiers. Please use `printf("%s", line);` — Weather Vane, Dec 20 '20 at 09:18
In addition to the [mcve], run your code through an autoindenter or autoformatter. You'll be surprised. — Ulrich Eckhardt, Dec 20 '20 at 09:19

score 2 · Answer 1 · edited Dec 20 '20 at 19:38

Use more C dynamic memory allocation.

Call stacks are typically limited to a megabyte and contain your automatic variables. Your call frame (for your array) is huge, and you probably have a stack overflow.

You need to use calloc(3) for your array and declare it as value*array; (of course, change the name, it becomes a pointer, not an array).

So

 value*ptr = calloc(60000, sizeof(value));
 if (!ptr) { perror("calloc failure"); exit(EXIT_FAILURE); };

then use ptr instead of array. Be aware that malloc(3) and calloc can fail. See also errno(3) and perror(3).

Don't forget to free(ptr) when it is not needed.

Read Modern C and take inspiration from existing open source software,

like GNU make or GTK. Read also some draft C standard, e.g. n1570.

Read also carefully the documentation of your C compiler, e.g. GCC and of your debugger, e.g. GDB. If you can use GCC, compile with gcc -Wall -Wextra -g to get several warnings, and perhaps even with -fanalyzer and other options, such as -Wstack-usage=2048

Consider using valgrind and the Clang static analyzer.

On Linux, strace(1) or ltrace(1) could be helpful, in addition of gdb(1). You could also consider using Frama-C.

I have code to sort 60000 lines ...

Then consider using qsort(3) or qsort_r

Read also a good operating system textbook.

PS. You might be interested in bismon (see this draft report) & in RefPerSys. In both cases, contact me by email (in English, or French, or Russian) to basile@starynkevitch.net

just tried what you proposed, the memory in stack has gone down. But new error :file access violation. — Dmitry, Dec 20 '20 at 09:01
Yes, you need to learn programming. Budget months of efforts. — Basile Starynkevitch, Dec 20 '20 at 09:07

score 1 · Answer 2 · answered Dec 20 '20 at 09:07

In addition to another answer, these lines (and the 5 other similar sets)

if (c == 0)
    strcpy(array[i].Time, p);
    array[i].Time = malloc(10000);

are lacking braces, are the wrong way round, and allocates excessive memory. Please change to

if (c == 0) {             // indentation alone does not make a code block
    array[i].Time = malloc(strlen(p) + 1);   // allocate the right amount
    strcpy(array[i].Time, p);                // AFTER allocation
}

Also these two lines

fgets(line, 60000, pointer); 
...
fgets(line, 60, pointer);

should both be

fgets(line, sizeof line, pointer);

How to prevent stackoverflow error when working with large array? C

2 Answers2

Use more C dynamic memory allocation.

Read Modern C and take inspiration from existing open source software,