An alert rule over time windows
Scripted traffic in 5-minute windows. Costs are in micro-dollars (1,000,000 = 1 dollar) so everything is a whole number. Both rules need 2 windows in a row.
A dashboard shows health at a glance, and an alert wakes a person only when something needs action. Good LLM alerts watch error rate, latency, cost anomalies and quality signals over time windows, and avoid firing on noise.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write fires(flags, n) where flags is a list of booleans (one per window) and the alert fires the first time n true values appear in a row. Return the 0-based index of that window, or -1 if it never fires.
Write cost_alert(costs, baseline, factor) returning the list of window indexes where the cost is above factor times baseline.
Explain it without notes
What is the difference between a dashboard and an alert, and what should wake someone up?
Why do alert rules use a minimum volume and a "for N windows" condition?
How would you detect a cost anomaly in an LLM app, and what would you put in place besides an alert?
Expected output
window 0: 20 requests, 5% errors, cost 8000
window 1: 22 requests, 4% errors, cost 9020
window 2: 20 requests, 10% errors, cost 7800
window 3: 21 requests, 42% errors, cost 8400
window 4: 20 requests, 50% errors, cost 8100
ALERT error-rate: true for 2 windows in a row
window 5: 20 requests, 5% errors, cost 30000
RESOLVED error-rate
window 6: 20 requests, 5% errors, cost 29600
ALERT cost-anomaly: true for 2 windows in a row
window 7: 20 requests, 5% errors, cost 8000
RESOLVED cost-anomaly