Logs & Basic Monitoring

In one line

tail -f, journalctl, and a handful of resource-monitoring commands are the difference between finding out about a problem from a user complaint versus catching it yourself first.

0/5 · 0%

Key ideas

01
tail -f <file> (follow) streams a log file live, printing new lines as they're written — the standard way to watch a service's behavior in real time while it's actively handling requests or, especially, right after a deploy, to confirm it started up cleanly.
02
journalctl (Phase 3's systemd logging) centralizes logs from every systemd-managed service in one place — journalctl -u <service> --since "1 hour ago" filters to a specific service and a specific time window, genuinely useful when investigating something that happened at a known approximate time rather than scrolling through everything.
03
journalctl -p err filters to only ERROR-level (and more severe) log entries across the WHOLE system — a fast first check when something's clearly wrong but you don't yet know which service is responsible, cutting through routine informational noise to just what's actually failing.
04
top/htop (Phase 3) show live CPU and memory usage; free -h shows a snapshot of memory specifically (used, free, and — importantly — how much is being used as disk CACHE, which the OS will happily give back to an application that actually needs it, so 'used' memory in free -h's naive sense often looks scarier than it actually is).
05
uptime shows how long the system's been running plus the LOAD AVERAGE (three numbers: average number of processes wanting CPU time over the last 1, 5, and 15 minutes) — a load average consistently higher than your machine's number of CPU cores suggests the system is genuinely CPU-constrained, not just busy.
06
Basic monitoring discipline: knowing WHERE to look (which log file, which service) BEFORE something breaks is what actually saves time during an incident — spending five minutes now confirming journalctl -u myapp -f shows sensible output is a cheap investment against a much more stressful, time-pressured search during a real outage later.

Code & diagrams

logs-and-monitoring.shmarkdown

The standard 'is everything okay' investigation sequence.

# Watch a log file live
tail -f /var/log/myapp/app.log

# A service's logs from the last hour, via systemd
journalctl -u myapp --since "1 hour ago"

# Only errors, system-wide, since this morning
journalctl -p err --since today

# Live resource usage
top
# or: htop

# Memory snapshot — note the "available" column, not just "used"
free -h

# System load and how long it's been running
uptime

Explain it without notes

Why can free -h's 'used' memory number be misleading if you don't also look at the cache/available figures?

What does it mean if uptime's load average is consistently higher than the number of CPU cores on the machine?

Practice

Start any service managed by systemd (or use an existing one) and use journalctl -u <service> -f to watch its logs live while it handles some activity.

Run free -h and uptime on any machine and interpret the numbers: is memory genuinely tight, and is the CPU under real load relative to its core count?

Trade-offs

↔
Command-line monitoring (top, free, journalctl) costs nothing to set up and works on any machine immediately, but it's fundamentally REACTIVE — you have to be actively looking at the right moment. A real production system typically layers dedicated monitoring/alerting (Prometheus, Grafana, a cloud provider's own monitoring) on top, specifically so problems surface proactively via an alert rather than requiring someone to happen to be watching a terminal when something breaks.

Done when you can

I can watch a log file live with tail -f and filter systemd logs by service and time window with journalctl.
I know free -h's 'available' figure is more meaningful than raw 'used' memory.
I can interpret uptime's load average relative to a machine's CPU core count.