Unit 1.1 · Databases in Production: PostgreSQL
PostgreSQL Essentials for DevOps
How Postgres is put together (processes, memory, WAL, MVCC), connecting and managing roles safely, the config files you'll touch, and where SQL vs NoSQL fits.
Start here
The mental model
Think of Postgres as a very careful librarian. Every change is first written in a journal (the WAL) before the shelves are touched. Nobody erases a page while someone might still be reading it; old versions are kept until no reader needs them (MVCC), and a cleaner (VACUUM) removes them later. Each visitor gets their own assistant (one process per connection), which is why too many visitors at once is a problem.
Go deeper
How it works inside
01Processes and memory
The postgres main process listens on port 5432 and FORKS one backend process per client connection. Background processes do the housekeeping: the WAL writer, checkpointer, background writer, autovacuum launcher, and WAL senders for replicas. Each connection costs several MB of RAM and some CPU scheduling overhead, which is why Postgres handles hundreds of connections well and thousands badly (Unit 1.2).
shared_buffers is Postgres's own page cache (typically ~25% of RAM); the OS page cache holds more. work_mem is per sort or hash operation, per connection, so 100 connections × 4 sorts × 64 MB could try to use 25 GB. Size it carefully.
02MVCC and VACUUM
Postgres never updates a row in place. An UPDATE writes a NEW row version and marks the old one dead; a DELETE just marks it dead. Readers see the version that was current when their transaction started, so readers never block writers. The cost: dead rows (BLOAT) accumulate until VACUUM reclaims the space. AUTOVACUUM does this automatically based on how many rows changed.
The classic outage: a transaction left open for hours ('idle in transaction', often a forgotten session or a stuck job) prevents vacuum from cleaning anything newer than it, so tables and indexes bloat and queries slow down. In extreme cases, transaction ID WRAPAROUND forces Postgres to stop accepting writes until an aggressive vacuum completes.
03Configuration and access
postgresql.conf holds the server settings; pg_hba.conf (host-based authentication) decides who may connect from where and how (scram-sha-256 passwords, certificates, IAM on RDS). On RDS these are PARAMETER GROUPS, and there is no superuser, only rds_superuser. SHOW setting; and the pg_settings view tell you what's actually in effect, and pending_restart shows changes waiting for a restart.
ROLES are users and groups. The least-privilege pattern: one owner role that runs migrations, one app role with only SELECT/INSERT/UPDATE/DELETE on the app schema, one read-only role for analysts and dashboards. Never let the app connect as postgres.
Do it
Hands-on lab
- 1
Start Postgres and look around
psqlmeta-commands start with a backslash:\llists databases,\dttables,\duroles, and\xtoggles expanded output.terminal$ docker run -d --name pg -e POSTGRES_PASSWORD=pw -p 5432:5432 postgres:17psql postgresql://postgres:pw@localhost:5432/postgres -c 'select version()' -c 'show shared_buffers' -c 'show max_connections'── expected output ──PostgreSQL 17.6 (Debian 17.6-1.pgdg120+1) on x86_64-pc-linux-gnu ...shared_buffers----------------128MBmax_connections-----------------100 - 2
Create least-privilege roles
The owner creates tables; the app role can only read and write data; default privileges make future tables follow the same rule.
roles.sqlwhole filesql CREATE DATABASE shop; \c shop CREATE ROLE shop_owner LOGIN PASSWORD 'owner-pw'; CREATE ROLE shop_app LOGIN PASSWORD 'app-pw' CONNECTION LIMIT 50; CREATE ROLE shop_ro LOGIN PASSWORD 'ro-pw'; CREATE SCHEMA shop AUTHORIZATION shop_owner; GRANT USAGE ON SCHEMA shop TO shop_app, shop_ro; ALTER DEFAULT PRIVILEGES FOR ROLE shop_owner IN SCHEMA shop GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO shop_app; ALTER DEFAULT PRIVILEGES FOR ROLE shop_owner IN SCHEMA shop GRANT SELECT ON TABLES TO shop_ro; REVOKE CREATE ON SCHEMA public FROM PUBLIC;terminal$ psql postgresql://postgres:pw@localhost/postgres -f roles.sqlpsql postgresql://shop_app:app-pw@localhost/shop -c 'create table shop.x(id int)'── expected output ──ERROR: permission denied for schema shop - 3
Watch MVCC create dead rows
Update every row once and the table doubles in dead tuples until vacuum runs.
pg_stat_user_tablesis where you watch bloat and autovacuum activity.terminal$ psql postgresql://postgres:pw@localhost/shop <<'SQL'create table t(id int, v int); insert into t select g, 0 from generate_series(1,1000000) g;update t set v = 1;select n_live_tup, n_dead_tup, last_autovacuum from pg_stat_user_tables where relname='t';vacuum t;select n_live_tup, n_dead_tup from pg_stat_user_tables where relname='t';SQL── expected output ──n_live_tup | n_dead_tup | last_autovacuum------------+------------+-----------------1000000 | 1000000 |n_live_tup | n_dead_tup------------+------------1000000 | 0 - 4
Find sessions blocking vacuum
pg_stat_activityis the first query in any Postgres incident: who is connected, what are they running, and for how long.terminal$ psql postgresql://postgres:pw@localhost/shop -c "select pid, usename, state, now()-xact_start as xact_age, left(query,40) from pg_stat_activity where state <> 'idle' order by xact_age desc nulls last limit 5"── expected output ──pid | usename | state | xact_age | left-------+----------+---------------------+-----------------+-----------------------------------------4127 | analyst | idle in transaction | 03:12:44.10912 | select * from orders where created_at >4190 | postgres | active | 00:00:00.00121 | select pid, usename, state, now()-xact_
Operate it
Knobs that matter
| Setting | Default | What it does | When to change it |
|---|---|---|---|
| shared_buffers | 128MB | Postgres's own buffer cache. | ~25% of RAM on a dedicated server (RDS sets this from instance size). |
| work_mem | 4MB | Memory per sort/hash operation before spilling to disk. | Raise carefully (16–64MB) if EXPLAIN shows external merge Disk; remember it's per operation per connection. |
| max_connections | 100 | Hard limit on client connections. | Keep in the low hundreds and use a pooler instead of raising it (Unit 1.2). |
| idle_in_transaction_session_timeout | 0 (off) | Kills sessions idle inside an open transaction. | Set to e.g. 5min to stop forgotten transactions from blocking vacuum and holding locks. |
| statement_timeout | 0 (off) | Cancels statements that run too long. | Set per role (ALTER ROLE shop_app SET statement_timeout='5s') so one bad query can't hog the database. |
| log_min_duration_statement | -1 (off) | Logs statements slower than N ms. | Set to 500ms–1s in production to catch slow queries in logs. |
3am practice
Failure drills
Each drill is a real failure mode. Read the scenario and the output, decide what's wrong, then reveal the diagnosis.
Drill #1
Queries slow down over a week, then recover after a restart
Response times grow steadily for days. Restarting the app 'fixes' it. The orders table is 5× larger on disk than its data.
Drill #2
Disk full, database read-only
A replication slot was created for a CDC tool that was later switched off. Weeks later the primary's disk fills up.
Decide
Relational vs the main NoSQL families
| Type | Examples | Great at | Trade-offs |
|---|---|---|---|
| Relational (SQL) | PostgreSQL, MySQL, Aurora | Transactions, joins, constraints, ad-hoc queries | Vertical scaling first; sharding is manual |
| Key-value | DynamoDB, Redis | Massive scale, predictable single-digit ms lookups | Access patterns must be designed up front; no joins |
| Document | MongoDB, DocumentDB | Flexible JSON documents, nested data | Multi-document transactions and joins are weaker/costlier |
| Wide-column | Cassandra, ScyllaDB | Huge write throughput, multi-region, always-on | Query by partition key only; eventual consistency to tune |
| Search | OpenSearch, Elasticsearch | Full-text search, log analytics, aggregations | Not a system of record; near-real-time, not transactional |
The bigger picture
Connects to
System Design · Transactions / ACID
Atomicity, Consistency, Isolation, Durability — the contract that makes bank transfers safe and your LLD's ledger correct.
System Design · Isolation Levels & Read Phenomena
Read Uncommitted → Read Committed → Repeatable Read → Serializable, and the dirty/non-repeatable/phantom reads each level tolerates.
Terraform · RDS Postgres
Provision the ShopLite database this unit operates.
AWS · RDS & Aurora
Parameter groups, Multi-AZ, storage, and IAM auth on the managed service.
Linux · Processes
Postgres's process-per-connection model is plain Linux fork().
Observability · N+1 queries
What a chatty app looks like from the database's side, in traces.
Prove it
Interview questions
What is MVCC and what problem does VACUUM solve?
How would you give an application least-privilege access to Postgres?