Topic 6B.8
Webhooks: Delivering Events to Other Systems
In one line
A webhook is an HTTP request your system sends to someone else's URL when something happens ('order delivered', 'payment captured'), instead of making them poll. Reliable webhooks are signed (so receivers know they're genuine), delivered at least once with retries and backoff, carry an event ID for deduplication, and are logged so partners can see and replay deliveries. Receivers should verify, store, acknowledge quickly, and process asynchronously.
Think of it like this
A courier ringing your doorbell when the parcel arrives, instead of you checking the porch every five minutes. But the courier should show ID (signature), come back if you're not home (retries), and not deliver the same parcel twice as if it were new (event IDs).
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Webhook
- An HTTP callback a system sends to a subscriber's URL when an event happens.
- HMAC signature
- A hash of the message made with a shared secret, proving it came from the holder of the secret and wasn't changed.
- Event ID
- A unique identifier for each event, used by receivers to ignore duplicates.
- Replay attack
- Resending a captured genuine request later. Timestamps in signatures limit it.
- Delivery log
- A record of every webhook attempt with its result, for debugging and replay.
Step by step
01Tiffin notifies a corporate partner
A company orders lunches for its employees through Tiffin and wants to know when each order is delivered, to update its own canteen app. Instead of polling Tiffin's API every minute for thousands of orders, it registers a webhook URL.
02Signing the payload
Tiffin signs the timestamp plus the raw body with the partner's secret. The partner recomputes the HMAC over exactly the bytes it received.
String sign(String secret, long timestamp, byte[] body) throws Exception {
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(secret.getBytes(UTF_8), "HmacSHA256"));
mac.update((timestamp + ".").getBytes(UTF_8));
byte[] sig = mac.doFinal(body);
return "t=" + timestamp + ",v1=" + HexFormat.of().formatHex(sig);
}
// Receiver: recompute, compare with MessageDigest.isEqual(...),
// and reject if |now - timestamp| > 5 minutes.03A well-behaved receiver
The partner's endpoint verifies the signature, inserts the event ID into a table with a unique constraint (duplicates are acknowledged and ignored), queues the work, and returns 200 straight away.
on POST /hooks/tiffin:
if !validSignature(rawBody, headers) or tooOld(timestamp): return 401
inserted = INSERT INTO webhook_events(id, body) VALUES (...) ON CONFLICT (id) DO NOTHING
if inserted: enqueue(processEvent, id)
return 200 # within a second or two, even if processing takes longerBreak it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Unsigned webhooks
A partner's endpoint accepts any POST with the right JSON shape, and Tiffin's webhooks aren't signed.
Myth vs fact
Myth
If the partner returned 200 once, they have the event.
Fact
They may have crashed before processing it. Partners need their own durable storage of events, and a way to fetch missed ones (an events API) as a backstop.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Offer an
GET /events?after=<cursor>API alongside webhooks. Partners can catch up after an outage or verify they didn't miss anything, and webhooks become a fast notification rather than the only delivery channel.
Remember this
- 1
Push, not poll: the provider calls
POST https://partner.example.com/hooks/tiffinwith a JSON event when something happens, saving partners from polling the API constantly. - 2
Signatures: sign the raw body with a shared secret (HMAC-SHA256) and include a timestamp, for example
Tiffin-Signature: t=1790850000,v1=<hex>. Receivers recompute and compare in constant time, and reject old timestamps to stop replays. - 3
At-least-once delivery: retry failed deliveries (timeouts, 5xx) with exponential backoff for hours or days. Duplicates will happen, so every event carries a unique
idthat receivers deduplicate on. - 4
Ordering isn't guaranteed: retries and parallel delivery can reorder events. Include a timestamp or sequence number and the resource's current state (or let receivers fetch the latest from the API).
- 5
Receivers should verify the signature, store the event, return 2xx within a few seconds, and do the real work asynchronously. Slow receivers cause timeouts and redeliveries.
- 6
Operations: a delivery log per partner (status, response code, attempts), manual replay, automatic disabling of endpoints that fail for days (with notice), and secret rotation with two valid secrets during the change.
Explain it without notes
Why must webhook receivers respond quickly?
Why include an event ID if every delivery is signed?
Practice
Design the retry schedule and disable policy for webhooks.
Verify a webhook signature in the language you use.
Trade-offs
- ↔
Webhooks are fast and efficient compared with polling, but delivery is at-least-once and unordered, and the sender must run retry infrastructure. Polling is simpler and receiver-controlled but wastes requests and adds delay.
Run it in production
You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:
Done when you can
I sign webhooks and verify them on receipt.
I deliver at least once with retries and event IDs.
My receivers store, acknowledge quickly, and process asynchronously.