Topic 14.3
Observability
In one line
Logs, metrics, traces — three signals, one question: 'what is happening and why?' (Prometheus, Grafana, OTel, ELK).
Think of it like this
A doctor being able to ask 'what were this patient's vitals for the last 24 hours' instead of only knowing 'the patient collapsed just now.' Logs, metrics, and traces are the vitals history for your software.
Key ideas
- 01
The three pillars: logs (narrative), metrics (numbers over time), traces (request causality across services).
- 02
Logs: structured JSON with traceId — searchable, sampled, tiered retention (12.21).
- 03
Metrics: RED (rate/errors/duration) per service, USE (utilization/saturation/errors) per resource; the golden signals (12.22).
- 04
Traces: OpenTelemetry spans = distributed request timeline — the only tool that answers 'which hop is slow'.
- 05
Correlation is the superpower: logs carry traceId/spanId, metrics carry labels — you go dashboard→trace→log in seconds.
- 06
Stack naming that interviewers expect: Prometheus + Grafana for metrics, ELK/OpenSearch for logs, OTel + Jaeger/Tempo for traces.
- 07
Rule of thumb: if you can't answer 'is it a problem or a blip?' from a dashboard, the answer to the interview question is observability.
Java / Spring map
- →
Micrometer + Prometheus registry, @Timed annotations, spring-cloud-sleuth-descendant OTel agent; SLF4J JSON layout with traceId.
Code & diagrams
One incident, three different questions — and each pillar answers a different one.
Explain without notes
A user reports 600ms latency; show the 90-second path: dashboard (metric) → trace → log — what does each layer reveal?
Practice
Design the golden-signal dashboards + one trace pipeline (OTel agent → collector → Tempo) for your service.
Trade-offs
- ↔
Full observability costs sampling decisions + storage money; over-instrument everything, then sample smart.
Run it in production
You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:
Completion checklist
I design with logs+metrics+traces correlated — and name the hole if one pillar is missing.