Command Palette

Search for a command to run...

Hectal
PHASE 10Advanced ~9 min· topic 1 of 4

Topic 10.1

Primary–Replica Replication

In one line

Replicas copy the primary asynchronously: a full sync with an RDB snapshot first, then a continuous stream of write commands. Replication IDs, offsets and a backlog let reconnecting replicas catch up with a partial resync instead of copying everything again.

0/4 · 0%

Think of it like this

A newsroom where one editor writes the paper and several copy desks retype it for regional editions. Each desk is slightly behind the editor. If a desk loses its connection briefly, it asks "what did I miss since line 5,210?" and catches up from the editor's recent notes; if it was gone too long, it needs a fresh copy of the whole paper.

Key ideas

  1. 01

    Setup: replicaof <primary-host> 6379 on the replica (or REPLICAOF at runtime). Replicas are read-only by default (replica-read-only yes).

  2. 02

    Full sync: the replica sends PSYNC ? -1; the primary replies FULLRESYNC <replid> <offset>, creates an RDB (to disk or streamed directly with diskless sync, the default since 7.0), and buffers new writes in the replica's output buffer. The replica loads the RDB and then applies the buffered and ongoing writes.

  3. 03

    Replication stream: after sync, every write command the primary executes is sent to replicas. Each side tracks a replication offset (bytes of stream processed). INFO replication shows master_repl_offset on the primary and each replica's offset and lag.

  4. 04

    Partial resync: the primary keeps recent stream bytes in the replication backlog (repl-backlog-size, default 1 MB, far too small for busy servers). A reconnecting replica sends PSYNC <replid> <offset>; if that offset is still in the backlog, it gets only the missing part (CONTINUE). Since Redis 4, replicas keep their replication ID across restarts and failovers (PSYNC2), so partial resync often works even after a promotion.

  5. 05

    Asynchronous means the primary acknowledges clients without waiting for replicas. Replica lag is usually milliseconds but grows under load, big writes, network issues or slow replicas.

  6. 06

    Sizing the backlog: backlog ≥ write rate (bytes/sec) × longest disconnect you want to survive. At 10 MB/s of writes and 60 s tolerance, use ~600 MB+, otherwise brief network blips trigger full syncs.

  7. 07

    Chained replication (replica of a replica) reduces load on the primary for many replicas. Replicas also run persistence and backups so the primary doesn't have to fork.

Code & diagrams

replication.redisredis
# On the replica
127.0.0.1:6380> REPLICAOF 10.0.0.1 6379
OK
# On the primary
127.0.0.1:6379> INFO replication
role:master
connected_slaves:2
slave0:ip=10.0.0.2,port=6379,state=online,offset=918273645,lag=0
slave1:ip=10.0.0.3,port=6379,state=online,offset=918271002,lag=1
master_replid:8f1c2d...
master_repl_offset:918273645
repl_backlog_active:1
repl_backlog_size:536870912
repl_backlog_first_byte_offset:381402734
# Replica log after a short network blip:
# Partial resynchronization accepted. Master replication ID: 8f1c2d... offset 918200000
psync.mermaiddiagram
Rendering diagram…

Interview problem

The problem

Replicas keep doing full resyncs

A 40 GB primary has two replicas. Every few hours a replica disconnects for ~20 seconds and then does a full resync, which saturates the network and sometimes loops. Explain why and fix it.

You're given

  • 40 GB dataset
  • Writes ~15 MB/s at peak
  • repl-backlog-size 1mb (default)

The interviewer follows up

01

Can replicas serve reads during a full resync?

When it breaks

Default 1 MB replication backlog on a write-heavy primary

What you see

Every short network blip becomes a full resync: high network use, primary fork, and replicas unavailable or stale for minutes.

Fix & prevent

Size the backlog from write rate × tolerated disconnect; watch sync_partial_err.

Explain it without notes

01

Explain replication ID, offset and backlog, and how they enable partial resync.

Practice

01

Set up a primary and replica in Docker, write continuously, pause the replica container for 10 s with a tiny backlog and then with a large one, and compare the logs.

Trade-offs

  • ↔

    Bigger backlogs cost primary memory but avoid expensive full syncs.

  • ↔

    Replicas add read capacity and failover targets, and double memory cost.

Done when you can

  • I can explain full sync, the replication stream and partial resync.

  • I can size the backlog and replica output buffers.