System 12.27 — Workflow Engine
In one line
The saga state machine as a product: durable steps, retries, human tasks, and the event-driven revival.
Think of it like this
A very reliable personal assistant that follows a checklist (approve → transfer money → notify), remembers exactly which step it was on if it gets interrupted, and picks up right where it left off.
Key ideas
- 01
Model: workflow = a graph of steps/states (approve → transfer → notify); state persisted per execution (durability!).
- 02
Execution: each step = a service call (or timer/human task); a crash must resume the exact step — state in DB, not memory.
- 03
Event-driven engines (Temporal-style): ever-running workflows that block on signals — step progress is driven by events, not polling.
- 04
The meat: durability (state per execution), retry policy per step, compensation (saga!), and human-in-the-loop tasks (approvals with deadlines).
- 05
Idempotency: steps re-run on retry → handlers must be idempotent (11.15).
- 06
Scale: many concurrent workflow instances × steps → a DB row per execution + a worker pool polling due steps.
- 07
Interviews: present it as 'sagas + durable state + human tasks, event-driven, with a coordinator' — recognition of the pattern pays.
Code & diagrams
State lives in the database, not in a running process — that's what lets a workflow survive a worker crashing mid-step.
Explain without notes
A worker dies mid-step-3. How does the engine know where to resume — and what if the step had already succeeded?
Practice
Design the workflow state table + the step retry/compensation table for loan processing.
Trade-offs
- ↔
Orchestrated (visible, central) vs event-driven (resilient, opaque) — the temporal-style hybrid is the modern answer.
Completion checklist
I can design a durable workflow engine with resume, retry, and compensation.