Malloc Memory Consumption Behavior in C

Question

I wrote this program to understand memory consumption pattern.

#include <stdio.h>
#include <stdlib.h>

int main()
{
   int i=0;

   while(1 < 2) {
   int *str = (int *) malloc(100000000);

   if(str == NULL) {
      printf("Out of memory.. %d", i);
      return(1);
   }
   printf("Attempt Count = %d\n", i++);

   }

   return(0);
}

In my system, I have 8 GB ram. Each malloc call will attempt to store 100MB. So I was expecting that my program would crash in (10* 8) loops. But the counter is going well beyond 600,000. I am not able to reason out the same. Kindly help.

Even considering all swap spaces and all, it should not increase more than 16GB, which does not look likely here.

It depends on the implementation of malloc(). On Linux, malloc overcommits. — P.P, Jun 16 '14 at 09:16

score 4 · Accepted Answer · edited May 23 '17 at 11:57

That's because in Linux the malloc'd memory is not used yet since you didn't initialize the memory.

You can malloc more than the memory you have (physical + virtual) because the kernel delays allocation of your memory until you actually use it. I believe that's to increase the chances of your program not failing due to memory limits, but that's not the question.

calloc is the same as malloc but zero initializes the memory. When you ask Linux for a page of memory, Linux already zero-initializes it. So if calloc can tell that the memory it asked for was just requested from the kernel, it doesn't actually have to zero initialize it! Since it doesn't, there is no access to that memory and therefore it should be able to request more memory than there actually is.

Check this answer. It provides a very good explanation.

Also the program wouldn't crash just by malloc'ing when it runs out of space it returns NULL.

I used `calloc` but still see the same results. – LPD Jun 16 '14 at 09:26 — LPD, Jun 16 '14 at 09:26

score 1 · Answer 2 · answered Jun 16 '14 at 09:20

In modern operating systems, the memory used by your process is virtual memory, it isn't limited by physical RAM size. It uses swap space on disk to allow for processes much larger than RAM.

In addition, some operating systems allow overcommitting. You can allocate more virtual memory than is actually available in swap space. Failures only occur when one of the processes actually tries to use too much of the memory.

Malloc Memory Consumption Behavior in C

2 Answers2