Segmentation fault: 11 with large array

Question

A simple program compiles fine but gives run-time error: segmentation fault: 11

int length=10000;
int num=100;
int num1=20;
int datablocklen=400002; //datablocklen=2*num1*length+2

int main(){
  double arr[num*length];
  double res[num][num];
  for(int i=0;i<num;i++){
    for(int j=0;j<num;j++){
      res[i][j]=0;
    }
  }
  for(int i=0;i<(num*length);i++){
    arr[i]=i;
  }
  int ntile=(int)(num/num1);
  double array_task[datablocklen];
  for(int i=0;i<ntile;i++){
    for(int j=0;j<ntile;j++){
    array_task[datablocklen-2]=i*num1*length;
    array_task[datablocklen-1]=j*num1*length;
      for(int k=0;k<(num1*length);k++){
        array_task[k]=arr[i*num1*length+k];
        array_task[num1*length+k]=arr[j*num1*length];
      }
    }
  }
return 0;
}

gcc -o test -std=c99 test.c to get the executable.

Strange still, the error does not show up if length is assigned a small value, say, 1000. But when it is larger than 10000, segmentation fault occurs.

Please note that I always keep an eye on the the value of datablocklen to make sure that datablocklen=2*num1*length+2. So if length or num1 changes, I will also change variable datablocklen.

I still have some problem dealing with gdb under mac, OS yosemite. So I have not debugged the program with gdb. But if I comment the inner for loop, the loop indexed by k, the program executes fine. I mean, no error message.

Computers might be powerful, but they do have limits. You can't just put in huge numbers and expect everything to run smoothly. If you really need to use huge numbers, try Ruby or Python 3. — Arc676, Dec 23 '15 at 06:33
@SouravGhosh I'm not sure about that. I have seen pages saying that this kind of error may be due to memory or swap limitation, or the stackoverflow thing. I don't know. — dudu, Dec 23 '15 at 06:36
Is a stack overflow, add `-fsanitize=address` to gcc and run it again. Also, I tested it with gcc optimizations `-O2` and it doesn't segv... gdb trace here http://pastebin.com/SyXWdrpt — Alex, Dec 23 '15 at 07:00
Also see http://stackoverflow.com/questions/22945647/why-does-a-large-local-array-crash-my-program — Lundin, Dec 23 '15 at 07:28

score 3 · Accepted Answer · edited Dec 23 '15 at 07:49

3

I think here stack memory is the issue here.

Two solutions:

Either increase the stack size for your process. Normally ulimit -s will show default size for any process in linux, mac
Instead of such large array allocation in stack double arr[num*length];use malloc for such memory allocation in heap.

edited Dec 23 '15 at 07:49

Jabberwocky

48,281
17
65
115

answered Dec 23 '15 at 06:36

Jeegar Patel

26,264
51
149
222

Usually it works just to move the large arrays outside of main, thus moving them from the stack. – Thomas Padron-McCarthy Dec 23 '15 at 07:05
@ThomasPadron-McCarthy yes good point. Global array would be stored in data segment and will be there until process terminate. While using malloc you have control of that memory and you can free when you want. – Jeegar Patel Dec 23 '15 at 07:10

Craig Estey · Answer 2 · 2015-12-23T07:22:34.463

You probably are overflowing the stack. It's default is usually around 8MB. Since your allocation sizes are fixed, you can move the big arrays to global space and switch many of your fixed values to defines. Here's an example [please pardon the gratuitous style cleanup]:

#define LENGTH          10000
#define NUM             100
#define NUM1            20
#define NTILE           (NUM / NUM1)

#if 0
#define DATABLOCKLEN    400002              // DATABLOCKLEN=2*NUM1*LENGTH+2
#else
#define DATABLOCKLEN    (2 * NUM1 * LENGTH + 2)
#endif

double arr[NUM * LENGTH];
double res[NUM][NUM];
double array_task[DATABLOCKLEN];

int
main()
{

    for (int i = 0; i < NUM; i++) {
        for (int j = 0; j < NUM; j++) {
            res[i][j] = 0;
        }
    }

    for (int i = 0; i < (NUM * LENGTH); i++) {
        arr[i] = i;
    }

    for (int i = 0; i < NTILE; i++) {
        for (int j = 0; j < NTILE; j++) {
            array_task[DATABLOCKLEN - 2] = i * NUM1 * LENGTH;
            array_task[DATABLOCKLEN - 1] = j * NUM1 * LENGTH;
            for (int k = 0; k < (NUM1 * LENGTH); k++) {
                array_task[k] = arr[i * NUM1 * LENGTH + k];
                array_task[NUM1 * LENGTH + k] = arr[j * NUM1 * LENGTH];
            }
        }
    }

    return 0;
}

You're welcome. FYI, I just sized your arrays and they totalled out at 11.2 MB — Craig Estey, Dec 23 '15 at 07:37

Segmentation fault: 11 with large array

2 Answers2