System 12.13 — Dropbox (Sync Engine)
In one line
The hard part isn't storage — it's reconciliation: chunked sync, delta algorithm, and conflict resolution.
Think of it like this
A filing cabinet that magically stays identical on your laptop, phone, and the cloud — the hard part is only sending the tiny CHANGED part of a file when you edit it, instead of re-uploading the whole thing every time.
Key ideas
- 01
Client ↔ server sync: chunked content-addressed storage — each file = list of chunk hashes; only NEW chunks upload.
- 02
Delta sync: client sends hashes of its chunks; server replies with missing ones → transfers shrink to the diff.
- 03
Content addressing (SHA-256) gives: dedupe (same chunk stored once), integrity (hash verifies), and resume.
- 04
Metadata DB: user → namespace → files(name, parent, version, chunk list, mtime) — the revision history tree.
- 05
Conflict resolution: last-writer-wins by default + 'sync conflict copy' on simultaneous edits; server is the arbiter (CRDTs exist as the advanced answer).
- 06
Consistency model: sequencing per file/user — a per-namespace monotonically increasing version (like a distributed log per user).
- 07
Notifications: long-poll/WS to tell a second device 'pull now' — plus polling fallback.
- 08
Interviews: the delta/rolling-hash (rsync-like) and conflict resolution are the differentiators.
Java / Spring map
- →
Metadata service (Postgres, per-user version counter); chunk blob store (S3); notification via WS/Kafka.
Code & diagrams
The sync engine's whole job: send only the bytes that changed, and resolve the conflict when two devices changed the same file offline.
Explain without notes
A 1GB file changes a 10KB middle chunk — trace exactly which bytes cross the network and why.
Practice
Design the per-file version table and the conflict rule with its 'conflict copy' behavior.
Trade-offs
- ↔
Chunk dedupe saves bandwidth+space vs smaller chunks = more metadata rows. 4MB chunks are the known balance.
Run it in production
You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:
Completion checklist
I can present chunked delta sync + content addressing + conflict versioning end-to-end.