why cant i manage ~382MB of memory when i have it available?

Question

OBJECTIVE: manage a unsigned long tomBOLA[5][10000000];

$top gives me:

top - 14:05:35 up  4:06,  4 users,  load average: 0.46, 0.48, 0.44
Tasks: 182 total,   1 running, 180 sleeping,   1 stopped,   0 zombie
Cpu(s): 14.4%us,  2.4%sy,  0.0%ni, 82.5%id,  0.6%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   3092064k total,  1574460k used,  1517604k free,   168944k buffers
Swap:  1998840k total,        0k used,  1998840k free,   672756k cached

program has, a malloc the size of (5*10000000) * 8bytes =382MB, then fill with 0s and a read of what is stored in tomBOLA:

long int **tomBOLA;

if((tomBOLA=(long int **)malloc(5))==NULL){ /*MALLOC()*/
    printf("\n\tMEMORY ERROR-1");
    exit(1);
}
for(i=0;i<5;i++){
    if((tomBOLA[i]=(long int *)malloc(10000000*sizeof(long int)))==NULL){
        printf("\n\tMEMORY ERROR-2");
        exit(1);
    }
}                                          /*MALLOC()*/
for(i=0;i<5;i++){
    for(j=0;j<10000000;j++){
        tomBOLA[i][j]=0;
    }
}                                         /*FILL WITH 0s before using*/
for(i=0;i<5;i++){
    for(j=0;j<10000000;j++){
        printf("%ld ",tomBOLA[i][j]);
    }
printf("\n\n\n\n\n");
}                                         /*CONTROL OF 0s*/

gdb gives

Program received signal SIGSEGV, Segmentation fault.
_int_malloc (av=<value optimized out>, bytes=<value optimized out>) at malloc.c:4708
4708    malloc.c: No such file or directory.
    in malloc.c
(gdb) bt
#0  _int_malloc (av=<value optimized out>, bytes=<value optimized out>) at malloc.c:4708
#1  0x001dfd4c in *__GI___libc_malloc (bytes=40000000) at malloc.c:3660
#2  0x0804ca86 in main ()
(gdb)

about memory, what am i doing or assuming wrong ??

The available memory doesn't matter really, as each process has its own virtual memory which the system will or will not put in the physical memory depending on how things are going. — Havenard, May 14 '13 at 13:35
@nhahtdh not possible, linux(ubuntu)+gcc+other malloc() worked fine so far — mf_, May 14 '13 at 13:38
@mf_: Now I am really curious why it can cause a segmentation fault in `malloc`. Of course, people can say accessing things out of bounds will give undefined behaviour, etc., but I still want to know what actually happens here... — nhahtdh, May 14 '13 at 13:45
@nhahtdh acording to @unwind was because i wasnt `tomBOLA = malloc(5 * sizeof *tomBOLA);` — mf_, May 14 '13 at 14:02
@mf_: I have read that before writing the comment. I don't understand why segmentation fault happens **inside** `malloc` function (I would expect segmentation fault to happen in your `main()` when you write to bad location). I do know about undefined behaviour and its definition, but I am curious about what is happening behind the scene. — nhahtdh, May 14 '13 at 14:06
i tried debugging and that was appeared to me, my version is `$ gdb -version GNU gdb (GDB) 7.1-ubuntu Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i486-linux-gnu".` — mf_, May 14 '13 at 14:14

score 7 · Accepted Answer · edited May 23 '17 at 12:13

7

This:

tomBOLA=(long int **)malloc(5))

is wrong, it's allocating 5 bytes, leading to immediate buffer overrun when you treat it as an array of five pointers (needing probably 20 bytes). You meant:

tomBOLA = malloc(5 * sizeof *tomBOLA);

to get five pointers to unsigned long.

Note that there's no point in doing this dynamically, it's way simpler to just say:

unsigned long *tomBOLA[5];

Also:

Get rid of the casts of malloc()'s return value.
Stop using magical constants (the solution above makes the 5 go away in the rest of the code, since you can then use sizeof tomBOLA / sizeof *tomBOLA in place of 5).

edited May 23 '17 at 12:13

Community

1
1

answered May 14 '13 at 13:35

unwind

391,730
64
469
606

1

@mf_ That's not the same thing. I realize you want five huge arrays, which is why you need five pointers. Five pointers is `unsigned long *tomBOLA[5];`, please note the asterisk in there. Then you can allocate each array with `malloc()`. – unwind May 14 '13 at 13:37
no casts, did as you recomended, works nice, but now i have a small problem, have to allocate manually other `unsigned long tomBOLA[OTHER_val][10000000];` – mf_ May 14 '13 at 14:00
Glad to hear it helped. Consider accepting the answer. I showed how to allocate a dynamic array of `unsigned long *`, that's the second code line: just do `unsigned long **arrays = malloc(n * sizeof *arrays);` where `n` is the number of arrays. – unwind May 14 '13 at 14:02

score 2 · Answer 2 · answered May 14 '13 at 13:37

Your first malloc is wrong. You need to allocate enough memory for the pointers.

if((tomBOLA=(long int **)malloc(5))==NULL){ /*MALLOC()*/

should be

if((tomBOLA=(long int **)malloc(5 * sizeof *tomBOLA))==NULL){ /*MALLOC()*/

(sizeof *tomBOLA here would be the same as sizeof(long int *) in this case, but it follows the general pattern of SomeType *p = malloc(sizeof *p))

score 0 · Answer 3 · answered May 14 '13 at 13:34

0

You may well have that memory available but not in a contiguous block which is what the malloc call is trying to give you.

answered May 14 '13 at 13:34

Bathsheba

231,907
34
361
483

alternatives to get a `unsigned long tomBOLA[5][10000000];` – mf_ May 14 '13 at 13:36
You can use a chained queue instead of an array. It will grow on demand and can get as big as a pointer can address. – Havenard May 14 '13 at 13:38

why cant i manage ~382MB of memory when i have it available?

3 Answers3