Health Probes & Graceful Shutdown

In one line

Readiness decides whether a pod gets traffic, liveness decides whether it gets restarted, startup protects slow starters. Combined with a preStop delay and SIGTERM handling, they make rollouts and scaling invisible to users.

0/5 · 0%

Think of it like this

A restaurant waiter. Readiness is 'I can take your table now' (don't seat guests with a waiter who's still setting up). Liveness is 'is this waiter frozen?' (replace them). Startup is 'new waiters get extra time on their first morning'. Graceful shutdown is finishing the current tables before going home.

Key ideas

01
READINESS probe failing → the pod is removed from Service endpoints (no traffic) but NOT restarted. Use it for 'warming up', 'overloaded', or 'my critical dependency is down'. LIVENESS probe failing → the kubelet kills and restarts the container. Use it ONLY for 'the process is stuck and can't recover' (a deadlock); never check dependencies in liveness, or a database blip restarts every pod at once. STARTUP probe → disables liveness/readiness until it succeeds once, protecting slow-booting apps (JVMs) from being killed during startup.
02
Probe types: httpGet (most common), tcpSocket, exec, and grpc. Tune periodSeconds, timeoutSeconds, failureThreshold; liveness should be more tolerant than readiness.
03
GRACEFUL SHUTDOWN: when a pod is deleted, Kubernetes simultaneously removes it from endpoints AND sends SIGTERM, but endpoint removal takes a moment to reach kube-proxy on every node and ingress controllers. Add a preStop sleep (5–10 s) so the pod keeps serving during propagation, then the app stops accepting new work, finishes in-flight requests, and exits before terminationGracePeriodSeconds (default 30 s), after which it's SIGKILLed.
04
Rolling updates depend on all of this: maxUnavailable: 0, maxSurge: 1 plus correct readiness means new pods take traffic only when ready and old pods drain cleanly (Phase 6, rolling updates).

In your stack

→
Spring Boot Actuator exposes /actuator/health/liveness and /actuator/health/readiness (enabled automatically on Kubernetes) and server.shutdown=graceful finishes in-flight requests on SIGTERM; keep spring.lifecycle.timeout-per-shutdown-phase below the grace period.

Code & diagrams

probes + graceful shutdownyaml

spec:
  terminationGracePeriodSeconds: 40
  containers:
    - name: api
      image: ghcr.io/shoplite/api:sha-3f2a1c9
      startupProbe:
        httpGet: { path: /actuator/health/liveness, port: 8080 }
        periodSeconds: 3
        failureThreshold: 40          # up to 120 s to start
      readinessProbe:
        httpGet: { path: /actuator/health/readiness, port: 8080 }
        periodSeconds: 5
        failureThreshold: 2
      livenessProbe:
        httpGet: { path: /actuator/health/liveness, port: 8080 }
        periodSeconds: 10
        failureThreshold: 6           # tolerant: only for real hangs
      lifecycle:
        preStop:
          sleep: { seconds: 10 }      # Kubernetes 1.30+; older: exec ["sh","-c","sleep 10"]

what happens when a pod is deleteddiagram

Rendering diagram…

Explain it without notes

Why must a liveness probe never check the database?

Practice

Every deploy causes a spike of 502s at the ingress for a few seconds. Diagnose and fix.

Trade-offs

↔
Aggressive liveness settings recover hung pods quickly but risk restart storms; conservative ones avoid storms but leave hung pods longer. Readiness is where most dependency logic belongs.

Done when you can

I can configure startup, readiness, and liveness probes for a JVM service
My rollouts drop zero requests

Command Palette