CloudWatch: Metrics, Logs & Alarms

In one line

CloudWatch collects metrics from every AWS service and your apps, stores and queries logs, and raises alarms — the default observability layer for anything running on AWS.

0/4 · 0%

Think of it like this

A car dashboard. METRICS are the gauges (speed, fuel), LOGS are the trip recorder with every event written down, and ALARMS are the warning lights that come on before the engine actually fails.

Key ideas

01
METRICS are time series organised by NAMESPACE (AWS/EC2, AWS/ApplicationELB), metric name, and DIMENSIONS (e.g. LoadBalancer, TargetGroup). AWS services publish many for free; memory and disk usage on EC2 require the CloudWatch AGENT. Custom metrics can be published via the API or, cheaply, as EMBEDDED METRIC FORMAT lines in logs.
02
LOGS live in LOG GROUPS (one per app/function) containing streams. Set a RETENTION period on every group — the default is 'never expire', which silently grows your bill. Log as structured JSON so LOGS INSIGHTS can filter and aggregate on fields.
03
ALARMS watch a metric over N evaluation periods and change state (OK/ALARM/INSUFFICIENT_DATA), triggering SNS notifications, Auto Scaling actions, or EC2 actions. Decide how missing data should be treated. COMPOSITE ALARMS combine several to reduce noise (e.g. page only if error rate AND latency are both bad).
04
Alarm on SYMPTOMS users feel — ALB 5xx rate, p99 latency (TargetResponseTime p99), queue age — rather than every CPU blip. Use PERCENTILES for latency, not averages: an average can look fine while 1% of users wait seconds.
05
Other pieces: DASHBOARDS for shared views, CONTAINER INSIGHTS and LAMBDA INSIGHTS for runtime metrics, SYNTHETICS canaries that call your endpoints on a schedule from outside, and ANOMALY DETECTION bands for metrics without a fixed threshold.

Code & diagrams

alarms.shbash

# Page when p99 latency on the ALB target group exceeds 1s for 3 of 5 minutes
aws cloudwatch put-metric-alarm --alarm-name api-p99-latency \
  --namespace AWS/ApplicationELB --metric-name TargetResponseTime \
  --dimensions Name=LoadBalancer,Value=app/prod-alb/abc Name=TargetGroup,Value=targetgroup/api/def \
  --extended-statistic p99 --period 60 --evaluation-periods 5 --datapoints-to-alarm 3 \
  --threshold 1 --comparison-operator GreaterThanThreshold \
  --treat-missing-data notBreaching \
  --alarm-actions arn:aws:sns:ap-south-1:123456789012:oncall

# Retention on every log group
aws logs put-retention-policy --log-group-name /ecs/api --retention-in-days 30

aws logs tail /ecs/api --follow --filter-pattern '{ $.level = "error" }'

insights-errors.sqlsql

Logs Insights over structured JSON logs: error count and p95 duration per route, 5-minute buckets.

fields @timestamp, route, status, durationMs
| filter status >= 500
| stats count(*) as errors, pct(durationMs, 95) as p95 by route, bin(5m)
| sort errors desc

emf-log-line.jsonjson

Embedded Metric Format: printing this log line also creates a custom metric — no PutMetricData calls.

{
  "_aws": {
    "Timestamp": 1758000000000,
    "CloudWatchMetrics": [{
      "Namespace": "Acme/Checkout",
      "Dimensions": [["PaymentProvider"]],
      "Metrics": [{ "Name": "PaymentLatency", "Unit": "Milliseconds" }]
    }]
  },
  "PaymentProvider": "razorpay",
  "PaymentLatency": 412,
  "orderId": "789"
}

Explain it without notes

Why alarm on p99 latency rather than average latency?

Your EC2 dashboards show CPU but no memory metric. Why?

Practice

Your team gets 40 alarm notifications a day and has started ignoring them. How do you fix the alerting?

Your CloudWatch Logs bill doubled. What do you check?

Trade-offs

↔
CloudWatch is built in and needs no infrastructure, but its costs rise with log volume and custom metric cardinality, and its query and dashboard tooling is less capable than dedicated stacks — many teams pair it with Prometheus/Grafana or a vendor for application observability.

Done when you can

I set retention on every log group and log structured JSON.
I alarm on user-facing symptoms using percentiles and M-of-N datapoints.
I can query logs with Logs Insights.
I install the CloudWatch agent where I need memory/disk metrics.