Command Palette

Search for a command to run...

Unit 5.1 · Performance Engineering

Load Testing Fundamentals with k6

The types of performance test, the numbers that matter (throughput, percentiles, errors), open vs closed workload models, coordinated omission, and a k6 test with thresholds that gates CI.

Beginner 50 min 5 lab steps 2 failure drills

Start here

The mental model

A load test is a rehearsal: you send fake customers at the system in a controlled way and watch what bends and what breaks. Each TYPE of test asks a different question. Can it handle normal peak traffic (LOAD)? Where does it break (STRESS/BREAKPOINT)? Can it survive a sudden rush (SPIKE)? Does it degrade over hours, from leaks or filling disks (SOAK)? Does the script even work (SMOKE)?

The answer is never one number. It's a curve: as load rises, latency stays flat, then bends upward sharply near saturation. Your job is to find where the knee is and keep production comfortably to the left of it.

Go deeper

How it works inside

01The numbers that matter

THROUGHPUT: requests (or transactions) per second completed. LATENCY as PERCENTILES: p50 is the typical user, p95/p99 are the unlucky ones, and averages hide them (Observability course, averages lie). ERROR RATE: a fast error is not a success. SATURATION of resources during the test: CPU, memory, connections, queue depth, GC. A result without the matching resource graphs tells you THAT it's slow, not WHY.

02Open vs closed models

A CLOSED model has a fixed number of virtual users (VUs), each sending a request, waiting for the response, then sending the next. When the system slows down, users send less, so the load politely drops, which real internet traffic never does. An OPEN model sends requests at a fixed ARRIVAL RATE (e.g. 200 per second) regardless of how slow responses are, like real users arriving independently. Use open models (k6 constant-arrival-rate / ramping-arrival-rate) to find true limits of public-facing services.

COORDINATED OMISSION: in a closed model, while the system is stalled, the tool isn't sending requests, so it doesn't record the requests that WOULD have been waiting. The measured p99 looks far better than reality. Arrival-rate executors avoid most of it.

Open vs closed modelsdiagram
Rendering diagram…

03Realistic tests

Model real user journeys (browse → search → add to cart → checkout) with realistic proportions and think time, use production-like data volumes (a test against an empty database proves nothing about indexes), randomise inputs so caches aren't artificially perfect, and test from outside the cluster through the real load balancer. Run against a production-like environment, never shared staging during someone else's demo, and tell the on-call engineers before you start.

04Performance tests in CI

A small smoke-load test on every merge with THRESHOLDS (p(95) < 300ms, errors < 1%) catches regressions early: the pipeline fails if a change makes checkout 40% slower. Bigger stress and soak tests run nightly or before launches. Canary analysis (GitOps course, Mission 2.2) is the production version of the same idea.

Do it

Hands-on lab

  1. 1

    A target to test

    Any HTTP service works. Grafana's QuickPizza demo app has a fast home page and a heavier recommendation API so you can see the difference.

    terminal
    $ docker run -d --name target -p 3333:3333 ghcr.io/grafana/quickpizza-local:latest
    curl -s -o /dev/null -w '%{http_code} %{time_total}s\n' localhost:3333/
    ── expected output ──
    200 0.004s
  2. 2

    Write a k6 test with stages and thresholds

    k6 scripts are JavaScript, but the engine is Go (no Node.js), and one machine can generate thousands of requests per second. The ramping-arrival-rate executor is an OPEN model. Thresholds make the run fail (non-zero exit code) when SLO-like targets are missed.

    loadtest.jswhole filejavascript
    import http from "k6/http";
    import { check, sleep } from "k6";
    
    export const options = {
      scenarios: {
        shoppers: {
          executor: "ramping-arrival-rate",
          startRate: 10, timeUnit: "1s",
          preAllocatedVUs: 50, maxVUs: 500,
          stages: [
            { target: 50,  duration: "1m" },   // warm up
            { target: 200, duration: "3m" },   // expected peak
            { target: 400, duration: "2m" },   // 2x peak: find the knee
            { target: 0,   duration: "30s" },
          ],
        },
      },
      thresholds: {
        http_req_failed:   ["rate<0.01"],               // < 1% errors
        http_req_duration: ["p(95)<300", "p(99)<800"],  // ms
        "http_req_duration{name:recommend}": ["p(95)<500"],
      },
    };
    
    const BASE = __ENV.BASE_URL || "http://localhost:3333";
    
    export default function () {
      const home = http.get(`${BASE}/`, { tags: { name: "home" } });
      check(home, { "home 200": (r) => r.status === 200 });
    
      const rec = http.post(`${BASE}/api/pizza`, JSON.stringify({ maxCaloriesPerSlice: 1000 }), {
        headers: { "Content-Type": "application/json", Authorization: "token abcdef0123456789" },
        tags: { name: "recommend" },
      });
      check(rec, { "recommend 200": (r) => r.status === 200 });
      sleep(Math.random() * 2);        // think time
    }
  3. 3

    Run it and read the summary

    Look at p95/p99 per tagged request, the failure rate, and dropped_iterations: if k6 couldn't start iterations at the requested rate (not enough VUs), the system is already saturated.

    terminal
    $ k6 run loadtest.js
    ── expected output ──
    █ THRESHOLDS
    http_req_duration
    ✓ 'p(95)<300' p(95)=212.4ms
    ✗ 'p(99)<800' p(99)=1.31s
    http_req_duration{name:recommend}
    ✗ 'p(95)<500' p(95)=742.8ms
    http_req_failed
    ✓ 'rate<0.01' rate=0.42%
     
    █ TOTAL RESULTS
    checks.........................: 99.58% 118412 out of 118912
    http_reqs......................: 118912 297.4/s
    dropped_iterations.............: 1843 4.6/s
    ERRO[0391] thresholds on metrics 'http_req_duration, http_req_duration{name:recommend}' have been crossed
  4. 4

    Stream results to Prometheus and Grafana

    Correlate load with resource graphs by sending k6 metrics to Prometheus (remote write) and viewing them next to your service dashboards (Observability course).

    terminal
    $ K6_PROMETHEUS_RW_SERVER_URL=http://localhost:9090/api/v1/write k6 run -o experimental-prometheus-rw loadtest.js
  5. 5

    Gate CI on the thresholds

    The job fails when thresholds fail. Keep CI runs short (1–2 minutes at modest load) and run big tests on a schedule.

    .github/workflows/perf.ymlwhole fileyaml
    name: perf-smoke
    on: [pull_request]
    jobs:
      k6:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v5
          - uses: grafana/setup-k6-action@v1
          - uses: grafana/run-k6-action@v1
            env:
              BASE_URL: ${{ vars.PREVIEW_URL }}
            with:
              path: tests/perf/smoke.js

Operate it

Knobs that matter

SettingDefaultWhat it doesWhen to change it
executorper-vu-iterationsHow k6 generates load.ramping-arrival-rate / constant-arrival-rate (open model) for services; VU-based for session-like workloads.
thresholdsnonePass/fail criteria.Derive from SLOs (SRE course): p95/p99 latency and error rate per important endpoint.
preAllocatedVUs / maxVUs—VU pool for arrival-rate executors.Enough that dropped_iterations stays 0 until the system itself saturates.
Load generator locationwherever you run itWhere traffic comes from.Outside the cluster, close to users' path; never on the same node as the target.

3am practice

Failure drills

Each drill is a real failure mode. Read the scenario and the output, decide what's wrong, then reveal the diagnosis.

Drill #1

Great results, terrible launch

The team load-tested checkout at 1,000 VUs: p99 120 ms. At launch, real traffic of ~300 req/s saw p99 of 4 s and timeouts.

terminal
$ cat old-test.js | grep -E 'executor|vus|sleep'
── what you'll see ──
executor: "constant-vus", vus: 1000,
sleep(5);

Drill #2

The load generator was the bottleneck

A stress test plateaus at exactly 2,100 req/s whatever the target, and server CPU is only 35%.

terminal
$ top -b -n1 | head -12 # on the load generator machine
── what you'll see ──
PID USER PR NI VIRT RES %CPU %MEM COMMAND
4121 ci 20 0 9.2g 3.1g 398.2 81.5 k6

Decide

Types of performance test

TestQuestion it answersShapeTypical length
SmokeDoes the script and system work at all?Tiny constant load1 min
LoadDoes it meet targets at expected peak?Ramp to peak, hold15–60 min
Stress / breakpointWhere and how does it break?Keep ramping until failureUntil it breaks
SpikeCan it absorb a sudden rush (sale, push notification)?Jump from low to very highMinutes
Soak / enduranceDoes it degrade over time (leaks, disk, connections)?Moderate constant load4–24 h

The bigger picture

Connects to

Prove it

Interview questions

01

What's the difference between load, stress, spike, and soak testing?

02

What is coordinated omission?

03

How do you make load tests realistic?

0/3 · 0%