Topic 10.10
Message Queue Basics
In one line
Producer → broker → consumer, and why the little words 'decouple', 'buffer', and 'retry' change everything.
Think of it like this
A to-do list pinned on a kitchen board. One person writes tasks on it (producer) whenever they think of them, and whoever is free later comes and picks off a task to do (consumer) — they don't have to be doing it at the exact same time.
Key ideas
- 01
Roles: Producer enqueues, Broker stores/routes, Consumer dequeues and processes.
- 02
What it buys: decoupling (producer doesn't wait), buffering (spike absorption), retry/redelivery, multiple consumers (fan-out), back-pressure.
- 03
Point-to-point (one consumer wins a message — RabbitMQ work queues) vs pub/sub (all subscribers get copies).
- 04
Delivery semantics: at-most-once (fire and forget / ack on send), at-least-once (ack after process → possible dupes), exactly-once (hard).
- 05
Consumer acking: auto-ack (fast, lose messages) vs manual ack (safer); dead letters on +retries overflow.
- 06
Interview line: 'I'll put this behind a queue so the producer never blocks and retries never hit the caller.'
Java / Spring map
- →
Spring AMQP (RabbitMQ): @RabbitListener, manual ack, TTL queues for delays.
Code & diagrams
The decision table for when a queue is the right answer.
Add a queue when:
- producer is slower than the consumer's rate → buffer (ratelimiter, fan-out)
- the request path doesn't need the result NOW → email, push, analytics
- upstream is unreliable → retry storage with DLQ
- two teams own producer/consumer → decoupled contracts
Don't queue when:
- the caller needs a synchronous answer (no async escape hatch)
- ordering must be global and strict (tricky across partitions)
- you're queueing just to hide a slow dependency (fix the dependency)Explain without notes
Ordering: two messages for the same order processed out of order — which queue property causes it, and what's per-partition order?
Practice
Put 'send payment receipt' behind a queue and specify the ack/failure path.
Trade-offs
- ↔
Queues trade consistency (eventual) for decoupling; your operational story needs monitoring + DLQ + replay.
Run it in production
You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:
Completion checklist
I can decide queue-or-not from latency/decoupling needs and specify ack semantics.