Measuring Delivery: DORA Metrics & Pipeline Observability

In one line

You can't improve delivery you don't measure. The four DORA metrics show outcomes; pipeline metrics (duration, queue time, failure and flaky rates, cost) show where the pipeline itself slows teams down.

0/4 · 0%

Think of it like this

A restaurant owner tracks not just food quality but how long orders take from ticket to table, how often dishes are sent back, and how quickly problems are fixed. Those numbers show where the kitchen needs help.

Key ideas

01
DORA METRICS (DevOps guide, fundamentals; Platform course, Mission 2.4): DEPLOYMENT FREQUENCY, LEAD TIME FOR CHANGES (commit to production), CHANGE FAILURE RATE (deployments needing remediation), FAILED DEPLOYMENT RECOVERY TIME. Compute them from your pipeline and deploy events plus incident data, per team, as trends.
02
PIPELINE METRICS: duration per stage (p50/p90), QUEUE TIME waiting for runners (a capacity signal for self-hosted runners), success/failure rate by cause (code vs infrastructure vs flaky), flaky test rate, re-run rate, and cost per pipeline minute. Slow or unreliable pipelines are one of the biggest developer-experience complaints.
03
OBSERVING PIPELINES: export CI events (GitHub Actions webhooks/API, GitLab analytics, Jenkins' Prometheus plugin) to dashboards; OpenTelemetry CI/CD semantic conventions and tools that trace pipelines as spans show exactly which step is slow. Alert on main being red for long and on runner queue time.
04
Use metrics for improvement, not ranking: Goodhart's law applies (Platform course, measuring the platform). Pair numbers with developer feedback.

Code & diagrams

pull pipeline durations from GitHubbash

gh run list --workflow release.yml --limit 100 --json databaseId,conclusion,createdAt,updatedAt \
  | jq -r '.[] | [.conclusion, ((.updatedAt|fromdate) - (.createdAt|fromdate))/60 | tostring] | @tsv' \
  | awk '{n[$1]++; t[$1]+=$2} END {for (c in n) printf "%-10s runs=%d avg=%.1f min\n", c, n[c], t[c]/n[c]}'
# success    runs=87 avg=9.4 min
# failure    runs=13 avg=6.1 min

Explain it without notes

Why track runner queue time separately from job duration?

Practice

Leadership asks, 'Is our CI/CD getting better?' Which five numbers would you show monthly?

Trade-offs

↔
Dashboards make bottlenecks visible but can be gamed if tied to targets; use them to find problems, and combine with developer surveys.

Done when you can

I can compute DORA metrics from pipeline and incident data
I monitor pipeline duration, queue time, and flakiness