Redis using 2GB of memory on 70MB data set - the fix

| Posted: - Last updated:
| Tagged redis memory malloc jemalloc
| ~400wrds (~2min)

For I use redis to cache pages and shards. This works great and all, but today I noticed something alarming:

redis	Running	3d 12h 4m 	0.0%	45.2% [1829556 kB]

Yes, that’s about 1.7 GB of RAM. That’s way too much for what I cache. Let’s see what redis has to say for itself:

$ redis-cli
redis> info

Well, that’s awkward. The OS is reporting 1.7GB memory usage, while Redis claims to store a mere 68MB. What’s happening here!

You may have noticed that I included mem_fragmentation_ratio in the snippet above as well. It’s at a whopping 26.16, meaning that for every byte I store, 26.16 bytes of memory are used. This explains the 1.7GB memory usage.

But, how do I get rid of this? My system has enough RAM to cope redis as is, but it’s not a comforting thought to leave Redis running like this.

As it turns out, there isn’t a lot you can do about this. Redis 2.2 uses malloc which causes the fragmentation. One alternative is to add a slave redis-server, migrate your data and then switch the slave to master. Although this is reported to work well, it’s not a good solution to the problem.

Fortunately, Redis 2.4 on Linux by default does not use malloc anymore. Instead it uses jemalloc. From the redis README:

Redis is compiled and linked against libc malloc by default, with the exception of jemalloc being the default on Linux systems. This default was picked because jemalloc has proven to have fewer fragmentation problems than libc malloc.

The only logical step to take is to upgrade to Redis 2.4.

If you’re already running (or just upgraded to) 2.4 you can easily check if your redis is using jemalloc:

$ redis-cli
redis> info

I’ve taken this step and the mem_fragmentation_ratio samples I’ve measured have all been in the 1.1-1.4 regions.

My site is free of ads and trackers. I record privacy-respecting usage statistics with Fathom.

Was this post helpful to you? Why not ☕ Buy me a coffee or use Brave Browser to send a tip.