Command Palette

Search for a command to run...

Hectal

Part 2

Kafka and Messaging

Event streaming with Kafka from first topic to production operations, and when a classic queue (RabbitMQ, SQS) is the better tool.

When ShopLite places an order, five things should happen: charge, reserve stock, send an email, update analytics, and index the order for search. Calling all five synchronously makes checkout as slow and fragile as the slowest one. Instead, checkout publishes one OrderPlaced event and each service reacts in its own time. Kafka is the most widely used platform for that, and it's also one of the most commonly misunderstood systems in production.

0/4 · 0%
  1. 2.1

    Kafka Fundamentals: Topics, Partitions, Offsets, Brokers

    Beginner

    The log as a data structure, how topics split into partitions, what an offset is, how brokers and KRaft controllers fit together, and your first producer and consumer.

    40 min · 4 lab steps · 2 drills

  2. 2.2

    Producers, Consumer Groups, Rebalancing, and Delivery Semantics

    Intermediate

    acks and idempotent producers, how consumer groups share partitions, what triggers a rebalance, committing offsets, at-least-once vs exactly-once, retries, dead-letter topics, and consumer lag.

    55 min · 4 lab steps · 2 drills

  3. 2.3

    Replication, ISR, Retention, Compaction, and Running Kafka

    Advanced

    How Kafka survives broker failure (replication factor, ISR, min.insync.replicas, unclean election), retention vs compaction, sizing partitions, schemas, and operating Kafka with Strimzi or MSK.

    60 min · 3 lab steps · 3 drills

  4. 2.4

    RabbitMQ, SQS/SNS, Redis Streams: Choosing a Broker

    Intermediate

    Queues vs logs, RabbitMQ exchanges and acknowledgements, SQS visibility timeouts and DLQs, SNS fan-out, Redis Streams, and a decision guide.

    45 min · 2 lab steps · 2 drills