Topic Strategy: One Topic, Many Topics, or Types in the Payload

In one line

Organise events as one topic per entity with multiple event types (ordering across types, union schemas), one topic per event type (clean schemas and ACLs, no cross-type order), or one catch-all topic (avoid). Decide by ordering needs, consumer filtering cost, retention and ownership.

0/4 · 0%

Think of it like this

Mail sorting. One mailbox per household keeps all of a family's letters together in arrival order; one mailbox per letter type (bills, postcards) is easier to process by type but loses the sequence between them.

Key ideas

01
Topic per entity (commerce.orders carrying OrderCreated, OrderPaid, OrderShipped): preserves per-order order across event types; consumers filter by type header; schema is a union (Avro union, Protobuf oneof) with RecordNameStrategy or a wrapper envelope.
02
Topic per event type (orders.created, orders.paid): simple schema per topic, precise ACLs and retention per type, consumers subscribe only to what they need; but no ordering between types for the same order.
03
One topic for everything: maximum filtering waste (every consumer reads everything), mixed retention, unclear ownership. Avoid except for firehose analytics ingest.
04
Retention differences push toward separate topics (audit events kept for years vs clickstream for days); ACLs push toward separate topics for sensitive events.
05
A pragmatic default: one topic per aggregate or entity for lifecycle events that must be ordered, separate topics for high-volume or differently retained streams.

Code & diagrams

envelope.prototext

syntax = "proto3";
package commerce.orders.v1;

message OrderEvent {
  string event_id = 1;
  string order_id = 2;
  int64  occurred_at_ms = 3;
  oneof payload {
    OrderCreated created = 10;
    OrderPaid    paid    = 11;
    OrderShipped shipped = 12;
  }
}

Explain it without notes

When would you split order events into per-type topics despite losing cross-type ordering?

Practice

Decide the topic layout for a ride-hailing app's trip events, driver locations and payment events.

Trade-offs

↔
Fewer entity topics keep order and reduce partitions; per-type topics simplify schemas, ACLs and retention.

Done when you can

I can choose a topic layout from ordering, filtering, retention and ownership needs.