Command Palette

Search for a command to run...

Hectal
on-callShift 3.3 · Operating at Scale
PLANNING

On-call review — Ravi was paged 31 times last week, 9 of them between midnight and 6 a.m.; he's said he wants to leave the rotation

Your teammate is burning out

You're the SRE lead reviewing on-call health with the team's manager.

sustainable on-callpaging budgetsescalation policiesfollow-the-sun and compensation

Briefing

The team has four people on a weekly rotation. Pages have doubled this quarter. Ravi's last week was the worst so far, and two others have mentioned exhaustion.

Your shift

Decide before you look. Each choice is locked once made; the next decision unlocks after it. There's no perfect path in real incidents either, only better and worse judgement under uncertainty.

Mon· decision 1 of 3

The manager suggests adding two more people to the rotation.

What's your first recommendation?

Debrief

Sustainable on-call is a reliability practice, not a perk. Reduce page volume at the source (delete, fix, or demote noisy alerts), set and track a paging budget, make escalation easy and expected, and share the load fairly, with compensation and time off after rough shifts.

The postmortem

Blameless postmortem · draft

Summary. On-call review led to alert cleanup, a paging budget, and a clear escalation policy.

Impact. Pages per week dropped from 31 to 9 within a month.

Timeline

  • MonPage audit: 3 alerts caused 58% of pages
  • TuePaging budget agreed (≤ 2 incidents/shift average)
  • WedEscalation policy published

Root cause. Unreviewed alert growth and a recurring bug generated most pages.

Contributing factors

  • No measurement of on-call load.
  • Cultural reluctance to escalate.

Action items

  • processWeekly on-call review: pages, causes, actionabilitysre
  • preventFix the recurring receipt-queue bug (see Shift 1.3)shoplite
  • processTime off in lieu after any night with more than 2 pagesmanager

The practice behind it

Sustainable on-call

Google's SRE practice suggests a small number of incidents per shift, enough to keep skills sharp but low enough to handle each one well and follow up properly. Too many pages cause fatigue and mistakes; too few mean skills fade. Rotations need enough people, handovers, clear escalation, and recognition such as pay or time off.

Your turn

01

Which three questions would you ask about every page in an on-call review?

Interview questions

01

How do you keep an on-call rotation healthy?

0/4 · 0%