Concurrent Cache

In one line

Building the Phase 10 cache-aside in-process first: thread-safe read/write, single-flight, and eviction under concurrency.

0/11 · 0%

Think of it like this

A popular ice cream shop with only one flavor machine. If 100 people ask for 'mango' at once and it's not ready, you don't send 100 people to fetch mangoes — one person fetches, and the other 99 just wait for that same batch.

Key ideas

01
ConcurrentHashMap + size bound is not LRU by itself: eviction needs an ordered structure.
02
Options: LinkedHashMap + lock, ConcurrentSkipListMap by timestamp, or a per-key write lock with a global eviction thread.
03
Single-flight: when the key misses, exactly ONE thread loads — others wait on its Future (cache stampede fix).
04
Stampede scenario: 10k threads miss, all hit the DB → the bug inside every naive cache.
05
Size/weight limits and periodic sweeps (expiry by TTL) must tolerate concurrent mutation.
06
Interview: mention stampede protection before you're asked — that's the senior signal.

Java / Spring map

→
Map<String, Future<V>> with computeIfAbsent = the classic single-flight implementation.

Code & diagrams

SingleFlightCache.javajava

Cache-aside with single-flight — the stampede fix.

public class LoadingCache<K, V> {
  private final ConcurrentHashMap<K, Future<V>> map = new ConcurrentHashMap<>();
  private final Function<K, V> loader;        // the expensive source (DB, API)

  public LoadingCache(Function<K, V> loader) { this.loader = loader; }

  public V get(K key) throws Exception {
    Future<V> f = map.computeIfAbsent(key, k -> {
      FutureTask<V> ft = new FutureTask<>(() -> loader.apply(k));
      ft.run();                                // start loading on this thread
      return ft;
    });
    return f.get();                            // others wait on the SAME future
  }
}
// 10,000 concurrent misses → ONE load. That is the cache-stampede fix.
// (Add eviction by removing the key in a sweeper or LRU layer separately.)

Explain without notes

Walk the interleaving without single-flight: how many DB queries on 10k same-key misses?

Practice

Add TTL eviction: a scheduled sweep that removes expired keys and a max-size LRU tier.

Trade-offs

↔
Single-flight holds waiting threads — if the loader hangs, everyone hangs; wrap load() with timeouts.

Run it in production

You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:

Stateful · Redis & caching patterns

Cache-aside with a stampede lock, TTLs, rate limiting, and sessions in Redis.

Stateful · Redis memory, eviction & persistence

maxmemory policies, big and hot keys, RDB/AOF, and slow commands.

Completion checklist

I include cache-through, TTL and stampede protection in any cache I design.

Back to phase

Command Palette