Measuring the Platform: DORA, Developer Experience, and Cost

1. 1

   Collect deployments and changes

   Apache DevLake (or Google's Four Keys, or a commercial tool) ingests GitHub, Argo CD, and incident data and computes DORA metrics out of the box. Whatever the tool, the model is three tables: CHANGES (commit SHA, time), DEPLOYMENTS (SHA, environment, time, success), and INCIDENTS (start, end, caused-by deploy). These queries show the calculations, so you understand the numbers the dashboards print.

   -- Deployment frequency: prod deploys per team per week
   SELECT team, date_trunc('week', deployed_at) AS week, count(*) AS deploys
   FROM deployments WHERE env = 'prod' GROUP BY 1, 2;
   
   -- Lead time for changes: median commit → prod deploy, last 30 days
   SELECT d.team,
          percentile_cont(0.5) WITHIN GROUP (ORDER BY d.deployed_at - c.committed_at) AS median_lead_time
   FROM deployments d JOIN changes c ON c.sha = ANY (d.shas)
   WHERE d.env = 'prod' AND d.deployed_at > now() - interval '30 days'
   GROUP BY d.team;
   
   -- Change failure rate: share of prod deploys linked to an incident or rollback
   SELECT team,
          round(100.0 * count(*) FILTER (WHERE caused_incident) / count(*), 1) AS change_failure_pct
   FROM deployments WHERE env = 'prod' AND deployed_at > now() - interval '90 days'
   GROUP BY team;
   
   -- Failed deployment recovery time: median incident duration for deploy-caused incidents
   SELECT team, percentile_cont(0.5) WITHIN GROUP (ORDER BY resolved_at - started_at) AS median_recovery
   FROM incidents WHERE caused_by_deploy IS NOT NULL GROUP BY team;

2. 2

   Install OpenCost

   Chart opencost from https://opencost.github.io/opencost-helm-chart, pointed at the cluster's Prometheus. Then query cost per team label.

   terminalcopy cmd

   $ kubectl -n opencost port-forward svc/opencost 9003:9003 &

   curl -s 'localhost:9003/allocation/compute?window=7d&aggregate=label:shoplite.dev/team' | jq -r '.data[0] | to_entries[] | "\(.key)\t$\(.value.totalCost|floor)"'

   ── expected output ──

   checkout $412

   storefront $298

   catalog $187

   fulfilment $96

   __idle__ $341

3. 3

   Give the platform SLOs

   Written with Sloth (or Pyrra), which generates Prometheus recording and burn-rate alert rules (Observability course, burn-rate alerts). Check the exact metric names your Backstage and Argo CD versions export (/metrics) before copying queries.

   version: prometheus/v1
   service: shoplite-platform
   slos:
     - name: golden-path-success
       objective: 99
       description: Backstage scaffolder runs that complete successfully.
       sli:
         events:
           error_query: sum(rate(scaffolder_task_count{result="failed"}[{{.window}}]))
           total_query: sum(rate(scaffolder_task_count[{{.window}}]))
       alerting: { name: GoldenPathFailing, page_alert: { disable: true } }
     - name: gitops-sync-success
       objective: 99.5
       description: Argo CD sync operations that succeed.
       sli:
         events:
           error_query: sum(rate(argocd_app_sync_total{phase!="Succeeded"}[{{.window}}]))
           total_query: sum(rate(argocd_app_sync_total[{{.window}}]))

4. 4

   Ask developers

   A five-question survey, every quarter, the same questions so trends are comparable. Free-text answers are the most valuable part: they become roadmap items.

   # ShopLite developer experience survey (5 minutes)
   
   1. How easy is it to get a change into production? (1 very hard – 5 very easy)
   2. How often did tooling or the platform block you this quarter? (never / monthly / weekly / daily)
   3. How confident are you when you deploy? (1–5)
   4. How easy was it to find the information you needed (docs, owners, runbooks)? (1–5)
   5. What single thing slowed you down most this quarter? (free text)

Make each change, run the command, and read the error before revealing the diagnosis. Recognising these messages on sight is what makes you fast on a real team. Undo the change afterwards.