Redis code reading: zmalloc

This post explains some fundamental functions of memory allocation used in Redis. These functions are kind of wrapper to hide architecture-dependent implementations and to keep track of the size of total allocated memory. The version of Redis is 3.0.5.

Available Backend Implementations

A backend memory allocator of zmalloc can be one of the following:

• libc malloc
  • Default except Linux.
• jemmaloc
  • http://www.canonware.com/jemalloc/
  • It seems less fragmentation than libc malloc. Default in Linux.
• TCMalloc (Thread-Caching malloc)
  • http://goog-perftools.sourceforge.net/doc/tcmalloc.html
  • It seems to be suited for multi-threaded program.

You can switch these backends at compile time.

zmalloc delegates an actual memory operation to a corresponding backend function and its primary role is to keep track of the size of allocated memory. Note that jemalloc, TCMalloc and default malloc in OS X already provide such a functionality.

Strategy to maintain allocated memory size

Here is the definition of zmalloc:

void *zmalloc(size_t size);

It returns a poinrter to an allocated memory as well as malloc and stores the total allocated memory. How to store it? It is very simple; just adds the size to a static global variable (with lock in thread-safe mode). However, this is insufficient because a size to subtract is unknown at memory deallocation.

Hence, Redis attaches size information at the prefix of every allocated memory.

As this figure shows, actual memory size is size+PREFIX_SIZE and the prefix holds the size. Tricky part is that a returned pointer addresses to the location just after the prefix. Like this, zmalloc behaves like malloc and can keep track of memory size.

Functions in zmalloc.h

Let’s take a look at some zmalloc functions. (Caution: following code is modified for explanation)

void *zmalloc(size_t size) {
    // allocate size+PREFIX_SIZE memory
    void *ptr = malloc(size+PREFIX_SIZE);

    // just handle OOM error
    if (!ptr) zmalloc_oom_handler(size);

    // store size to the prefix
    *((size_t*)ptr) = size;

    // add allocated size to a static global variable
    update_zmalloc_stat_alloc(size+PREFIX_SIZE);

    // return the pointer just after the prefix
    return (char*)ptr+PREFIX_SIZE;
}

void zfree(void *ptr) {
    if (ptr == NULL) return;

    realptr = (char*)ptr-PREFIX_SIZE;

    // restore the size in prefix
    oldsize = *((size_t*)realptr);

    // subtract allocated size from a static global variable
    update_zmalloc_stat_free(oldsize+PREFIX_SIZE);

    // free entire memory
    free(realptr);
}

It is very straightforward! Several other functions are provided in zmalloc.c:

• zcalloc, zrealloc :: wrappers to calloc, realloc respectively
• zstrdup :: duplicate an allocated memory
• zmalloc_size :: return size of an allocated memory
• zmalloc_used_memory :: return total allocated memory size

These are simple and easy to read, too (zrealloc is little difficult though).