Command Palette

Search for a command to run...

Hectal
Roadmap
Phase 13BAdvanced17 of 19 in the curriculum

Data-Intensive Building Blocks

The internals senior engineers reach for: storage engines (B-tree vs LSM), unique ID generation, Bloom filters and sketches, geospatial indexes, clocks and ordering, gossip and Merkle trees, stream vs batch processing, and backpressure.

Phase 12's systems keep reusing the same small set of clever building blocks: a Bloom filter in the crawler, a geohash in Uber, a Snowflake ID in Twitter, an LSM tree in Cassandra. This phase explains each one properly, so you can use them deliberately instead of name-dropping them.

0/8 · 0%
8 topics ~57 min 8 code blocks & diagrams
Start with the first topic
1
13B.1

Storage Engines: B-Trees, LSM Trees & the WAL

Under every database is a storage engine that decides how bytes hit the disk. B-trees update data in place and excel at reads; LSM trees append and merge later and excel at writes. Knowing which one you're on explains most performance behaviour.

7 min 1 diagram practice

2
13B.2

Unique ID Generation at Scale

Auto-increment IDs need one central counter; random UUIDs scatter indexes. Distributed systems use time-ordered IDs (Snowflake, UUIDv7, ULID) that are unique without coordination and sort by creation time.

8 min 1 code practice

3
13B.3

Bloom Filters, HyperLogLog & Count-Min Sketch

When exact answers are too big or too slow, probabilistic structures give almost-right answers in tiny memory: 'definitely not present' (Bloom filter), 'about 48 million unique visitors' (HyperLogLog), 'roughly how often' (Count-Min Sketch).

7 min 1 code practice

4
13B.4

Geospatial Indexing: Geohash, Quadtrees & H3

'Find drivers within 2 km' can't scan every driver. Geospatial indexes turn 2-D locations into cells or keys you can look up quickly: geohash strings, quadtrees that split busy areas, and hexagonal grids like H3.

7 min 1 code practice

5
13B.5

Time, Clocks & Ordering: Lamport, Vector Clocks, HLC

Machines' clocks disagree, so 'which happened first?' can't be answered with timestamps alone. Logical clocks capture cause and effect; vector clocks detect concurrent conflicting writes; hybrid clocks combine both with real time.

7 min 1 diagram practice

6
13B.6

Gossip, Failure Detection & Merkle Trees

Large clusters can't have every node ping every other node. Gossip spreads membership and state epidemically, phi-accrual detectors decide who's dead, and Merkle trees let replicas find differences by comparing a few hashes.

7 min 1 diagram practice

7
13B.7

Batch vs Stream Processing, OLTP vs OLAP

Operational databases serve the app (OLTP); analytics needs different storage (OLAP). Batch jobs process large bounded datasets periodically; stream processing handles unbounded events continuously. Most companies run both, connected by change data capture.

7 min 1 diagram practice

8
13B.8

Backpressure, Load Shedding & Admission Control

When demand exceeds capacity, a system must slow producers down, reject some work gracefully, or collapse. Bounded queues, backpressure, load shedding by priority, and adaptive concurrency limits keep it standing.

7 min 1 diagram practice