10

I would like to use a high performance general purpose allocator like jemalloc/tcmalloc with a memory pool. Is there a guide for doing this? I don't want to use jemalloc/tcmalloc as a drop-in replacement for malloc.

I have memory pool that uses libarena and carves a largish memory area (2GB). I want to create fixed size objects pool like gslice on this arena for say 1 GB. (like the Bonwick slab allocator) and want to use the rest of the memory for variable sized general purpose allocation. I would like to have jemalloc/tcmalloc use this memory area. Is this possible? Any ideas as to how I can proceed? If there is an alternate library, I would like to hear about it as well.

Martin Prikryl
  • 188,800
  • 56
  • 490
  • 992
John Knight
  • 109
  • 1
  • 3
  • 2
    Good luck getting this answered, I have also really wanted this answered, please see my http://stackoverflow.com/questions/23341587/malloc-like-function-using-custom-heap own question. – Vality Aug 04 '14 at 13:05
  • I wrote an allocator specifically for our app. One thing that turned out to be a real nifty idea was the concept of reducing all possible allocations to a small number of "quantum" sizes. Below 4k = 8 bytes, 4k..16k = 16 byte... up to 64mb max. I ended up with only had about 1000 different sizes all the way up to 64 MB. Wasted ram was low since the size of the wasted space was a function of the size of the allocation. This allowed recycle piles for each size. A free was just a push and an alloc was a pop, unless the pile was empty for that size. – johnnycrash Mar 11 '15 at 20:01
  • I don't have an answer for you, but you may find some useful info on this jemalloc thread regarding allocating memory as additional arenas for custom allocators here: http://www.canonware.com/pipermail/jemalloc-discuss/2015-January/000988.html – acanaday Apr 11 '15 at 22:04

1 Answers1

8

This is an old question, but there's finally a positive answer, at least where jemalloc is concerned. Since jemalloc version 4.0.0, you can use mallctl to set hooks for where chunked allocations get memory by using the arena.<i>.chunk_hooks interface. These hooks are well documented in the jemalloc manpage, including a number of cross-references.

I'm unaware as to any similar interface in tcmalloc; I do not use it.

I answered a similar question semi-recently; there may be useful information there, as well.

dho
  • 2,310
  • 18
  • 20