Topic 17.5
40 Hands-On Builds, Study Plans, and the Readiness Checklist
In one line
Build the 40 projects in five levels, follow the plan for your experience, and use the checklist before claiming "I know Kafka". Each build should include tests for crashes, duplicates and rebalances, which become your interview stories.
Think of it like this
Learning to drive happens on the road, not from the manual. These builds are your road hours.
Key ideas
- 01
Level 1: basic producer, basic consumer, multiple consumer groups, key-based partitioning, manual offset commits, consumer lag monitor, producer retry handling, JSON event pipeline.
- 02
Level 2: order event pipeline, payment consumer, inventory consumer, notification consumer, retry topic, dead-letter topic, idempotent consumer, partition-aware consumer.
- 03
Level 3: transactional producer, exactly-once Kafka-to-Kafka pipeline, Streams aggregation, Streams join, tumbling, sliding and session window analytics, stateful Streams app.
- 04
Level 4: transactional outbox, inbox pattern, CDC pipeline, Schema Registry integration, Connect pipeline, multi-tenant Kafka, multi-region Kafka, cross-region DR. Level 5: production-grade event platform, payment event platform, e-commerce saga, real-time analytics platform, CDC platform, event-sourcing platform, CQRS platform, global event streaming platform.
- 05
Study plans: 0–2 years: basics → topics → partitions → producers → consumers → groups → offsets → basic retries and ordering (Phases 0–3, 6). 2–4 years: add replication, ISR, acks, min.insync, idempotence, rebalancing, lag, DLT, Schema Registry, Connect, Streams (Phases 4–11). 4–7 years: internals, KRaft, storage, performance, capacity, hot partitions, EOS, state stores, cross-region DR, security, observability (Phases 10–14). Staff: consensus concepts, failure domains, multi-region, event sourcing, CQRS, saga, outbox/inbox, CDC, EOS boundaries, backpressure, SLOs, RPO/RTO, cost, governance, multi-tenancy, and when not to use Kafka (Phases 13–17).
Code & diagrams
[ ] Architecture: broker, topic, partition, offset, replica, leader, ISR, controller, KRaft
[ ] Keys and partitioning, ordering scope, partition count planning, hot partitions
[ ] Producer: batching, compression, acks 0/1/all, retries, idempotence, max.in.flight
[ ] Consumer: poll loop, commits (auto/manual/sync/async), lag, groups, rebalancing
[ ] Rebalance protocols: eager, cooperative, static membership, KIP-848
[ ] Delivery semantics, transactions, read-process-write, EOS boundaries, inbox
[ ] Replication factor, min.insync.replicas, unclean election, rack awareness, AZ survival
[ ] Segments, indexes, page cache, zero-copy, retention, compaction, tombstones, tiered storage
[ ] Retries, backoff, jitter, retry topics, DLT, poison messages, backpressure
[ ] Schemas: Avro, Protobuf, JSON Schema, registry, BACKWARD/FORWARD/FULL
[ ] Connect, CDC with Debezium, outbox
[ ] Streams: KStream, KTable, GlobalKTable, state stores, changelogs, windows, event time, joins
[ ] Security (TLS, SASL, ACLs), quotas, multi-tenancy, capacity planning, large messages
[ ] Monitoring (URP, under-min-ISR, offline, lag), debugging trees, failure injection
[ ] DR: MirrorMaker 2, offset translation, RPO/RTO; ZooKeeper-to-KRaft migration
[ ] Saga, event sourcing, CQRS, replay and backfill
[ ] Kafka vs Redis, RabbitMQ, SQS; anti-patterns; when NOT to use KafkaInterview problem
The problem
Your 8-week Kafka plan
You have 8 weeks before senior backend interviews. Plan Kafka preparation using this course and the 40 builds.
Explain it without notes
Apply the reasoning chain to "design a payment event pipeline" in one breath.
Practice
Complete at least one build per level and write a one-page failure report for each.
Done when you can
I've built at least one project per level with failure tests.
I can tick every readiness item without notes and answer designs with the full reasoning chain.