2

I am trying to read a data file containing about 10^7 values and perform some calculations. I am creating an array of that size and doing fscanf to read those values into each of the elements of the array. The gist of the program looks like this

#include<stdio.h>
#include<math.h>

int main()
{
    int L = 10000000;
    float array[L];
    FILE *fp;
    fp = fopen("datafile.txt","r");
    /*
    reading values into the array from datafile.txt using fscanf
    and doing some operations on array elements
    */
    fclose(fp);
    return 0;
}

But the same program works if I use L of lesser magnitude, i.e for L=10^6 and lesser it works fine.
Firstly I thought I have lesser primary memory on my laptop (~4GB) then I tried to run the program on high-end computers having 16GB and 128GB main memory there also I got segmentation fault(core dumped)

I used gcc to compile the program which compiled the program without any error and warnings.

gcc my_program.c -lm
./a.out

The output was segmentation fault as I mentioned.

Galilean
  • 246
  • 4
  • 15
  • 5
    The array is too big for the stack. Move the array outside of `main`, or declare the array as `static`. – user3386109 Oct 19 '18 at 19:59
  • is there any upper bound of declaring the array? what do you mean by stack here? @user3386109 – Galilean Oct 19 '18 at 20:02
  • 3
    https://en.wikipedia.org/wiki/Call_stack The upper bound is determined by the stack size, which varies between implementations. I generally try to avoid arrays bigger than a few K bytes. A third option (in addition to the two I mentioned in the first comment) is to allocate memory for the array using `malloc`. – user3386109 Oct 19 '18 at 20:09
  • 1
    the posted code does not compile! Amongst other problems, this statement: `fp = fopen("datafile.txt",'r');` has the wrong syntax. the second parameter is expected to be a pointer to a string, I.E. `fp = fopen("datafile.txt","r");` Note the use of double quotes rather than single quotes. your compiler should have told you about this problem. When compiling, always enable the warnings, then fix those warnings. ( for `gcc`, at a minimum use: `-Wall -Wextra -Wconversion -pedantic -std=gnu11` ) Note other compilers use different options to perform the same things – user3629249 Oct 19 '18 at 21:49
  • 1
    OT: when calling the function: `fopen()`, always check (!=NULL) the returned value to assure the operation was successful – user3629249 Oct 19 '18 at 21:51
  • At least you came to the right place for your `StackOverflow`... – David C. Rankin Oct 19 '18 at 23:02

1 Answers1

4

You're probably blowing your stack. For anything "big" allocate dynamically using something like calloc:

int main()
{
    int L = 10000000;
    float *array = calloc(L, sizeof(float));

    FILE *fp;
    fp = fopen("datafile.txt",'r');
    /*
    reading values into the array from datafile.txt using fscanf
    and doing some operations on array elements
    */
    fclose(fp);

    free(array);
    return 0;
}

Local variables are limited in size, so trying to create a local variable that's "too big" will result in unpredictable behaviour or even crashes. The memory remaining for local variables depends on how deeply nested your code is, so it can fluctuate wildly. This is why keeping your local variables to a minimum is important. Pointers and integers are really cheap, but arrays of a consequential size are trouble.

tadman
  • 208,517
  • 23
  • 234
  • 262
  • 2
    Maybe `calloc(L, sizeof *array)` would be better? Only a nitpick. – Osiris Oct 19 '18 at 20:05
  • 1
    @Osiris I don't see why that would be better. It just looks strange. – tadman Oct 19 '18 at 20:26
  • 1
    A lot of C code is predicated on being able to pay attention to such things, so it's usually not an issue. C++ on the other hand with `auto` is a whole other deal. – tadman Oct 19 '18 at 20:57
  • 2
    Yes i agree with you, I just wanted to point out that there is an option which would be safer against changing types (really just a nitpick), even in C. It is a good answer though. – Osiris Oct 19 '18 at 21:41
  • @Osiris Options are always educational. There's pros and cons here, like with everything. – tadman Oct 19 '18 at 21:45
  • Yes of course, i only wanted to mention it. BTW I did not DV. – Osiris Oct 19 '18 at 21:48
  • 2
    In this example where the allocation and the declaration are in the same statement, perhaps it does not seem important, but where the declaration and the allocation occur separately, possibly many lines apart, it is far better from a maintainability point of view to use the type of the object rather then an explicit type, so it is a useful habit to get into. It does not look strange at all - it is good practice. I see only pros; what do you think are the cons? Bare in mind that the developer and the maintainer may not be the same person, or same skill level, – Clifford Oct 19 '18 at 23:03