Unit 3.2 · Redis and Caching

Memory, Eviction, Persistence, and Slow Commands

maxmemory and eviction policies, big keys and hot keys, RDB snapshots vs AOF, fork and copy-on-write, the SLOWLOG, and the commands you should never run in production.

Intermediate 45 min 4 lab steps 2 failure drills

Start here

The mental model

Redis keeps everything in RAM, and RAM is finite. When it's full, Redis must either refuse new writes or throw something away. Which one you want depends on whether Redis is a CACHE (throw away the least-used keys, since they can be rebuilt) or a STORE (sessions, queues, locks: never silently drop data, fail loudly instead).

Persistence is optional in Redis. With it off, a restart means an empty Redis. With it on, Redis periodically photographs memory to disk (RDB) and/or keeps a journal of every write (AOF).

Go deeper

How it works inside

01maxmemory and eviction policies

maxmemory caps data size; set it below the container or instance memory to leave room for overhead, fragmentation, and fork copy-on-write (see below). When the cap is reached, maxmemory-policy decides: noeviction (the default) returns errors on writes; allkeys-lru / allkeys-lfu evict the least recently / least frequently used key of any kind (best for pure caches); volatile-lru / volatile-ttl evict only keys that have a TTL (for mixed cache + store use, which usually means you should split it into two instances).

Watch evicted_keys and used_memory in INFO. A cache with a steadily rising eviction rate and falling hit ratio (keyspace_hits / (hits + misses)) is too small.

02Big keys and hot keys

A BIG KEY (a 50 MB string, a hash with 5 million fields) is slow to read, delete (DEL frees memory synchronously, so use UNLINK), replicate, and migrate, and it blocks the event loop while doing so. A HOT KEY (one key receiving a large share of all traffic) is limited by a single thread on a single node, even in a cluster. Find them with redis-cli --bigkeys, --memkeys, and --hotkeys (with an LFU policy). Fix big keys by splitting (cart:{user}:items per user instead of one global hash), and hot keys with a local in-process cache or replicas for reads.

03RDB and AOF

RDB: Redis fork()s a child process that writes a compact snapshot of memory to dump.rdb (on a schedule, or BGSAVE). Fast to restart from and great for backups, but you lose everything since the last snapshot. The fork relies on COPY-ON-WRITE: pages the parent modifies during the snapshot are duplicated, so a write-heavy Redis can need up to ~2× memory briefly. If there isn't enough, the fork fails or the OOM killer strikes (Docker course, cgroups).

AOF: every write command is appended to a log; appendfsync everysec (the default) loses at most about 1 second on a crash. The AOF is periodically rewritten (compacted) in a background fork too. Common production setup for data you care about: AOF everysec plus periodic RDB for backups. For a pure cache: often neither, and accept a cold start.

RDB and AOFdiagram

Rendering diagram…

04Slow and dangerous commands

KEYS pattern walks every key and blocks Redis; use SCAN with a cursor. FLUSHALL/FLUSHDB do what they say. SMEMBERS, HGETALL, and LRANGE 0 -1 on huge collections are O(n); use SSCAN/HSCAN or ranges. SLOWLOG GET shows commands over slowlog-log-slower-than microseconds, and LATENCY DOCTOR gives a human-readable report. Disable or rename dangerous commands with ACLs (ACL SETUSER app -@dangerous) so an application user can't run them.

Do it

Hands-on lab

1
Fill a small Redis and watch eviction
Cap memory at 20 MB with an LRU policy, write 100,000 keys of ~1 KB, and look at the eviction counter.
terminal
$ rc CONFIG SET maxmemory 20mb && rc CONFIG SET maxmemory-policy allkeys-lru
docker exec redis redis-benchmark -t set -n 100000 -r 100000 -d 1000 -q
rc INFO stats | grep -E 'evicted_keys|keyspace_(hits|misses)'; rc DBSIZE
── expected output ──
SET: 88495.58 requests per second, p50=0.271 msec
evicted_keys:81342
keyspace_hits:0
keyspace_misses:0
(integer) 16791
2
Now with noeviction
The same load against a store-style config fails loudly instead of losing data.
terminal
$ rc CONFIG SET maxmemory-policy noeviction
rc SET one-more-key x
── expected output ──
(error) OOM command not allowed when used memory > 'maxmemory'.
3
Find big keys and slow commands
Create a big set, scan for big keys, then run a slow command and read the slowlog.
terminal
$ rc FLUSHALL && rc CONFIG SET maxmemory 0
rc EVAL "for i=1,300000 do redis.call('SADD','big:set',i) end" 0
docker exec redis redis-cli --bigkeys | grep -A1 'Biggest'
rc SMEMBERS big:set > /dev/null; rc SLOWLOG GET 2
── expected output ──
Biggest set found "big:set" has 300000 members
...
1) 1) (integer) 3
2) (integer) 1790461234
3) (integer) 38211 <- 38 ms blocking every other client
4) 1) "SMEMBERS"
2) "big:set"
4
Turn on AOF and inspect persistence
INFO persistence shows whether the last save and rewrite succeeded, which is the thing to alert on.
terminal
$ rc CONFIG SET appendonly yes && rc CONFIG SET appendfsync everysec
rc BGSAVE; sleep 2; rc INFO persistence | grep -E 'rdb_last_bgsave_status|aof_enabled|aof_last_write_status|rdb_last_cow_size'
── expected output ──
rdb_last_bgsave_status:ok
rdb_last_cow_size:1265664
aof_enabled:1
aof_last_write_status:ok

Operate it

Knobs that matter

Setting	Default	What it does	When to change it
maxmemory	0 (no limit)	Data size cap.	Set to ~60–75% of available RAM if persistence forks; higher for no-persistence caches.
maxmemory-policy	noeviction	What to do when full.	`allkeys-lfu`/`allkeys-lru` for caches; `noeviction` for stores (and alert on memory).
appendonly / appendfsync	no / everysec	AOF journal and how often it fsyncs.	`yes` + `everysec` for data you need; `always` is safest but slow.
save	3600 1 300 100 60 10000	RDB snapshot schedule.	Disable (`save ""`) on pure caches; keep for backups otherwise.
lazyfree-lazy-user-del	no (yes in some 8.x defaults)	Make DEL free memory in the background like UNLINK.	Enable to avoid stalls when deleting big keys.
slowlog-log-slower-than	10000 µs	Threshold for the slowlog.	Lower to 1000 µs to catch more; export slowlog length as a metric.

3am practice

Failure drills

Each drill is a real failure mode. Read the scenario and the output, decide what's wrong, then reveal the diagnosis.

Drill #1

Redis pod OOMKilled during snapshots

Redis runs in Kubernetes with limits.memory: 4Gi and holds 3.2 GB of data with RDB enabled. It restarts every few hours, always around save time.

terminal

$ kubectl describe pod redis-0 | grep -A3 'Last State'

── what you'll see ──

Last State: Terminated

Reason: OOMKilled

Exit Code: 137

Drill #2

Latency spikes every few minutes

Redis p99 latency is 0.3 ms normally but jumps to 800 ms every few minutes. CPU and memory look fine.

terminal

$ rc SLOWLOG GET 1

── what you'll see ──

1) 1) (integer) 991

2) (integer) 1790462110

3) (integer) 781332

4) 1) "KEYS"

2) "session:*"

5) "10.0.3.17:52144"

6) "session-cleanup-cron"

The bigger picture

Connects to

System Design · Eviction Policies: LRU, LFU, TTL

When the cache is full, what leaves?

Docker · cgroups

Why exceeding a memory limit means OOMKilled (exit 137).

Observability · Memory leak case

Telling a real leak from expected cache growth using metrics.

Unit 0.1 · Durability

AOF everysec is the same safety-vs-speed trade-off as fsync.

Prove it

Interview questions

What eviction policy would you use for a cache vs a session store?

RDB vs AOF?

Why should you never run KEYS in production?

0/3 · 0%

Command Palette