The kubectl Debugging Toolkit

In one line

describe, logs, exec, port-forward, and events cover the overwhelming majority of real troubleshooting — knowing exactly which one answers which question is most of what separates fast debugging from guessing.

0/4 · 0%

Key ideas

01
This topic assembles the full toolkit Phase 0.4 introduced individually, now specifically organized around WHICH QUESTION each command actually answers — the fastest debugging happens when you go straight to the right tool for the specific question, rather than trying commands somewhat randomly.
02
'Why won't this even schedule/start?' → kubectl describe pod <name>, reading its Events section — this answers scheduling failures, image pull failures, and volume mount failures, all of which happen before your application code ever runs at all (Topic 6.1).
03
'What did my application actually do/print?' → kubectl logs <name> (current run) or kubectl logs --previous <name> (the run before its last restart) — this answers application-level problems, once you've confirmed via describe that the container did actually start running in the first place.
04
'Is this actually a networking problem?' → kubectl exec -it <pod> -- sh to get an interactive shell INSIDE the container, then use ordinary tools from there (curl, nslookup, wget) exactly as Linux's own course taught — genuinely useful for confirming whether a dependency is reachable from the pod's own actual point of view, rather than guessing from outside it.
05
'I need to reach this pod/service directly from my own laptop, right now, without exposing it properly' → kubectl port-forward <pod-or-service> <local-port>:<remote-port> — temporarily tunnels a local port on your own machine directly to a port inside the cluster, genuinely useful for quick, ad-hoc debugging (hitting a database or an internal API directly) without needing to set up a real Ingress or LoadBalancer just to look at something once.
06
'What has happened to this resource recently, across its whole history, not just right now?' → kubectl get events --sort-by=.lastTimestamp (cluster or namespace-wide) or the Events section within describe (scoped to one resource) — genuinely useful for spotting a PATTERN of recurring problems (repeated restarts, repeated scheduling failures) that a single point-in-time snapshot wouldn't reveal on its own.

Code & diagrams

DebuggingQuestionMapdiagram

Match the question you're actually asking to the right command — this is most of the skill.

Rendering diagram…

full-toolkit.shmarkdown

A realistic, complete debugging session working through the toolkit in order.

# 1. Did it even start?
kubectl describe pod my-app-7f8b9

# 2. If it started, what did it actually do?
kubectl logs my-app-7f8b9
kubectl logs --previous my-app-7f8b9

# 3. Is a dependency actually reachable from this pod's own point of view?
kubectl exec -it my-app-7f8b9 -- sh
# inside the container:
# curl -v http://payments-svc:8080/health
# nslookup payments-svc

# 4. I want to hit the database directly from my own laptop, just once
kubectl port-forward svc/postgres-svc 5432:5432
# now, from another terminal on your own machine:
# psql -h localhost -p 5432 -U myuser mydb

# 5. Has this been happening repeatedly, or is it a one-off?
kubectl get events --sort-by=.lastTimestamp -n production | tail -20

Explain it without notes

You suspect a pod can't reach a database it depends on. Walk through which specific commands you'd use, in order, to confirm or rule this out.

Why is kubectl port-forward described as being for 'quick, ad-hoc' debugging rather than a real, ongoing way to expose a service?

Practice

Pick any running pod you have access to and walk through describe, logs, and exec in sequence, even if nothing is actually wrong — build the muscle memory before you need it under real pressure.

Use kubectl port-forward to reach a Service running in a cluster directly from your own machine, confirming you can interact with it locally without needing any Ingress or LoadBalancer set up.

Trade-offs

↔
This handful of commands covers the overwhelming majority of real troubleshooting, but they're all REACTIVE — you have to already know something's wrong and go looking. A mature production setup pairs this manual toolkit with proactive observability (metrics, centralized logging, alerting) so problems surface on their own rather than requiring someone to think to run kubectl describe in the first place — this manual toolkit remains essential even then, for the actual investigation once something's been flagged.

Done when you can

I can match a specific troubleshooting question to the right kubectl command without guessing.
I can use kubectl exec to test networking/connectivity from inside a pod's own point of view.
I know kubectl port-forward is for temporary, ad-hoc access, not a real production exposure mechanism.