Sharding

In one line

Partitioning across separate physical databases — the real 'put more money in the machine' of writes.

0/10 · 0%

Think of it like this

A huge chain of supermarkets where each city has its OWN separate warehouse and inventory system, instead of one giant central warehouse for the whole country. It's the difference between 'a bigger room in the same building' (partitioning) and 'a whole separate building' (sharding).

Key ideas

01
Shard = an independent full database holding a subset of rows; the app (or proxy) routes by shard key.
02
Why: write throughput of one DB is finite; sharding splits the write fan-out across machines.
03
Shard key rules the world: pick one dimension queries actually use (user_id, org_id, geo).
04
Cross-shard problems: joins die, transactions die (unless 2PC), unique index scope shrinks per shard.
05
Common keys: user_id-hash (social), geo (regional services), tenant (SaaS).
06
Interviews love the catch: 'every query must carry the shard key — otherwise you scan every shard'.

Java / Spring map

→
Shard routing via tenant/user id in a middleware or via separate DataSources; Vitess/ShardingSphere as managed options.

Code & diagrams

routing by shard keydiagram

Rendering diagram…

ShardRouter.javajava

The routing decision is the whole art.

public class ShardRouter {
  private final Shard[] shards;                    // 0 .. N-1

  public ShardRouter(int shardCount) {
    shards = new Shard[shardCount];
    for (int i = 0; i < shardCount; i++) shards[i] = new Shard(i);
  }

  public Shard shardFor(String userId) {
    int slot = Math.floorMod(userId.hashCode(), shards.length);   // stable, even
    return shards[slot];
  }

  // Every query carries the key → routes to ONE shard.
  // Cross-shard smell: "give me all orders" → scatter-gather across ALL shards.
  public List<Order> ordersFor(String userId) {
    return shardFor(userId).query("orders WHERE user_id = ?", userId);
  }
}

Explain without notes

Design shards for a chat app. What is the shard key, and which popular query becomes a scatter-gather?

Practice

Pick the shard key for payments, then defend why your 'list all transactions of a user' stays single-shard.

Trade-offs

↔
Sharding is the last scaling lever for a reason: it costs joins, transactions, and operational sanity.

Run it in production

You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:

Stateful · Redis Sentinel, Cluster & locks

Hash slots, failover, and why distributed locks need fencing.

Stateful · Indexing, mappings & shards

Inverted indexes, analyzers, and shards in OpenSearch.

Completion checklist

I can pick a shard key and name the query that breaks.

Back to phase

Command Palette