2

I want to test how much the OS does allocate when I request 24M memory.

    for (i = 0; i < 1024*1024; i++)
            ptr = (char *)malloc(24);

When I write like this I get RES is 32M from the top command. 

    ptr = (char *)malloc(24*1024*1024);

But when I do a little change the RES is 244. What is the difference between them? Why is the result 244?

Christian Rau
  • 45,360
  • 10
  • 108
  • 185
Fei Xue
  • 1,995
  • 5
  • 19
  • 29
  • 4
    In short: One leaks memory and the other (maybe) doesn't. – Mysticial Jul 05 '12 at 07:46
  • 1
    Don't type-cast the return value of malloc. See [Do I cast the result of malloc?](http://stackoverflow.com/questions/605845/do-i-cast-the-result-of-malloc) – LihO Jul 05 '12 at 07:48
  • 6
    ...and the second creates 1 contiguous allocation, while the first may create numerous small allocations (and quite possibly introduce additional fragmentation, per-allocation overhead, etc.)... – reuben Jul 05 '12 at 07:48
  • @reuben, should have made that as an ans. – tuxuday Jul 05 '12 at 07:55
  • A parenthesis: both examples will crash and burn on a 8 or 16-bit computer, because of integer overflow. – Lundin Jul 05 '12 at 09:52

3 Answers3

7

The allocator has its own data structures about the bookkeeping that require memory as well. When you allocate in small chunks (the first case), the allocator has to keep a lot of additional data about where each chunk is allocated and how long it is. Moreover, you may get gaps of unused memory in between the chunks because malloc has a requirement to return a sufficiently aligned block, most usually on an 8-byte boundary.

In the second case, the allocator gives you just one contiguous block and does bookkeeping only for that block.

Always be careful with a large number of small allocations, as the bookkeeping memory overhead may even outweigh the amount of the data itself.

Blagovest Buyukliev
  • 42,498
  • 14
  • 94
  • 130
6

The second allocation barely touches the memory. The allocator tells you "okay, you can have it" but if you don't actually touch the memory, the OS never actually gives it to you, hoping you'll never use it. Bit like a Ponzi scheme. On the other hand, the other method writes something (a few bytes at most) to many pages, so the OS is forced to actually give you the memory.

Try this to verify, you should get about 24m usage:

memset(ptr, 1, 1024 * 1024 * 24);

In short, top doesn't tell you how much you allocated, i.e. what you asked from malloc. It tells you what the OS allocated to your process.

cnicutar
  • 178,505
  • 25
  • 365
  • 392
0

In addition to what has been said: It could be that some compilers notice how you allocate multiple 24 Byte Blocks in a loop, assigning their addresses to the same pointer and keeping only the last block you allocated, effectively rendering every other malloc from before useless. So it may optimize your whole loop into something like this:

ptr = (char *)malloc(24);
i = 1024*1024;