System 12.16 — Payment System (HLD)
In one line
The highest-stakes design: money invariants, idempotency, gateway adapters, reconciliation, webhooks.
Think of it like this
The Phase 4 payment LLD scaled to a whole company's worth of traffic: same 'never double-charge' rule, now spread across a fleet of servers that must all agree on the outcome of every single transaction.
Key ideas
- 01
Money rules (repeat them): cents everywhere, idempotency keys, unique index on (order, key), no floats, audit log.
- 02
Layers: client → API service → payment service (ledger + state machine) → gateway adapters → PSPs (Razorpay/Stripe) → webhook ingestion.
- 03
State machine: INITIATED → PROCESSING → SUCCEEDED/FAILED (retryable transient) / REFUNDED / DISPUTED — transitions guarded + logged.
- 04
Idempotency: /charge carries key; unique index returns the memoized result on duplicate — the code is in Phase 4.
- 05
Ordering guarantee: charge-then-fulfillment = saga; payment success must trigger fulfillment EXACTLY ONCE (outbox → consumer idempotent).
- 06
Reconciliation: daily batch reconciliation against PSP settlement files — the 'trust but verify' layer every payment company has.
- 07
Webhooks: PSP outcome callbacks must be idempotent, signature-verified, and processed via a queue (never inline).
- 08
Ledger/journal: double-entry writes per movement (debit/credit) — the accounting core; never mutate, always append.
- 09
Consistency: payments demand strong consistency for balance reads (read-your-writes) + eventual for notifications.
- 10
Security: PCI-DSS scope, tokenization (card → token at PSP), mTLS to PSP, secrets in Vault, auditability everywhere.
Java / Spring map
- →
@Transactional local ledger writes; outbox publisher; gateway port + adapters; webhook signature filter.
Code & diagrams
One rule drives every box in this diagram: never let two paths both think they charged the card.
Explain without notes
The duplicate-charge scenario through LB retry + gateway retry + consumer restart — where each duplicate dies.
Practice
Design the daily reconciliation job: sources, matching key, exception queue, alert rules.
Trade-offs
- ↔
Strong consistency + audit = latency + complexity — the price of money. Never trade idempotency for speed.
Run it in production
You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:
Completion checklist
I can present payments with invariants, idempotency, saga, reconciliation, and PCI posture.