Mission 4.1 · Stage 4 — Data & Secrets
RDS Postgres in the Database Subnets
Goal: A private, encrypted Postgres 16 instance in the database subnets, reachable only from the app security group, with its master password generated and stored by AWS — not by you.
By the end of this mission
- Create a DB subnet group, parameter group, and RDS instance
- Set production-vs-dev behaviour (Multi-AZ, deletion protection, final snapshots) from variables
- Let RDS manage the master password in Secrets Manager
- Connect to a private database through Session Manager port forwarding
Part 1
Understand it first
What Terraform owns vs what the database owns
Terraform manages the database INSTANCE: size, storage, network placement, backups, encryption, parameters. It does not manage what's INSIDE the database (tables, rows, application users). Schemas are an application concern, handled by migrations that ship with the app. Mixing them (running SQL from Terraform) couples schema changes to infrastructure applies and is rarely worth it.
Never let Terraform know the password
The classic mistake is password = var.db_password: the password is then in tfvars or CI variables, in every plan's inputs, and in state in plain text forever. manage_master_user_password = true asks RDS to generate the password, store it in Secrets Manager, and rotate it on a schedule. Terraform only ever sees the secret's ARN. Mission 4.2 wires that ARN into ECS.
Dev and prod differ in the dangerous settings
Settings like deletion_protection, skip_final_snapshot, multi_az, and apply_immediately are where environments should differ. Dev: single-AZ, no deletion protection, no final snapshot, apply changes immediately, so it's cheap and disposable. Prod: Multi-AZ, deletion protection on, a final snapshot on delete, and changes held for the maintenance window. Deriving them from var.environment means prod can't accidentally get dev's settings.
Pinning only the major version
With auto_minor_version_upgrade (the default), AWS upgrades 16.4 to 16.6 during maintenance windows. If your code says engine_version = "16.4", the next plan tries to DOWNGRADE, which fails. Specifying just "16" means 'any 16.x'. The provider compares against the major version, and engine_version_actual tells you the running minor.
Part 2
Your project after this mission · 3 files change
- infra/
- alb.tf
- autoscaling.tf
- backend.tf
- database.tfnew
- ecr.tf
- ecs.tf
- iam.tf
- locals.tf
- logs.tf
- network.tf
- outputs.tfmodified
- probe.tf
- providers.tf
- refactors.tf
- security.tf
- storage.tf
- terraform.tfvars
- variables.tfmodified
- versions.tf
Part 3
Build it, step by step
- 1
Database variables
Only size and HA are variables; the safety settings are derived from the environment in the resource itself.
infra/variables.tfadd to filehcl variable "db_instance_class" { type = string default = "db.t4g.micro" } variable "db_multi_az" { description = "Synchronous standby in a second AZ. Required in prod." type = bool default = false } - 2
Subnet group and parameter group
The subnet group tells RDS which subnets it may use, here the database tier with no internet route. The parameter group forces TLS for every connection (
rds.force_ssl) and logs queries slower than 500 ms. Parameter groups can't be renamed or have their family changed in place, hencename_prefixwithcreate_before_destroy, the same pattern as security groups.infra/database.tfwhole filehcl resource "aws_db_subnet_group" "main" { name = "${local.name_prefix}-db" subnet_ids = [for s in aws_subnet.database : s.id] } resource "aws_db_parameter_group" "postgres16" { name_prefix = "${local.name_prefix}-pg16-" family = "postgres16" parameter { name = "rds.force_ssl" value = "1" } parameter { name = "log_min_duration_statement" value = "500" } lifecycle { create_before_destroy = true } } - 3
The instance
Read it top to bottom: encrypted gp3 storage that autoscales up to 100 GB;
manage_master_user_passwordinstead of a password; placed in the database subnets with the DB security group from Mission 2.3; not publicly accessible; seven days of point-in-time backups; and the environment-dependent safety settings.infra/database.tfadd to filehcl locals { is_prod = var.environment == "prod" } resource "aws_db_instance" "main" { identifier = "${local.name_prefix}-db" engine = "postgres" engine_version = "16" instance_class = var.db_instance_class allocated_storage = 20 max_allocated_storage = 100 storage_type = "gp3" storage_encrypted = true db_name = "shoplite" username = "shoplite_admin" manage_master_user_password = true db_subnet_group_name = aws_db_subnet_group.main.name vpc_security_group_ids = [aws_security_group.db.id] parameter_group_name = aws_db_parameter_group.postgres16.name publicly_accessible = false multi_az = var.db_multi_az backup_retention_period = 7 copy_tags_to_snapshot = true deletion_protection = local.is_prod skip_final_snapshot = !local.is_prod final_snapshot_identifier = "${local.name_prefix}-db-final" apply_immediately = !local.is_prod } - 4
Outputs — an ARN, not a password
master_user_secretis a list with one element holding the secret's ARN. That's all Terraform knows.infra/outputs.tfadd to filehcl output "db_endpoint" { value = aws_db_instance.main.address } output "db_master_secret_arn" { value = aws_db_instance.main.master_user_secret[0].secret_arn } - 5
Apply — and be patient
RDS creation takes 5–10 minutes. Terraform polls and prints progress. Everything else in the plan finishes in seconds.
terminal$ terraform apply── expected output ──Plan: 3 to add, 0 to change, 0 to destroy....aws_db_instance.main: Still creating... [5m0s elapsed]aws_db_instance.main: Still creating... [7m30s elapsed]aws_db_instance.main: Creation complete after 7m52s [id=db-ABCDEFGHIJKLMNOPQRSTUVWXYZ]Apply complete! Resources: 3 added, 0 changed, 0 destroyed.Outputs:...db_endpoint = "shoplite-dev-db.c1x2y3z4a5b6.ap-south-1.rds.amazonaws.com"db_master_secret_arn = "arn:aws:secretsmanager:ap-south-1:123456789012:secret:rds!db-7f3e...-AbCdEf" - 6
Connect through a Session Manager tunnel
The database has no public endpoint and only accepts the app security group. The probe instance from Mission 2.4 uses that group, so turn it on and forward a local port through it to RDS. No SSH, no bastion, no open ports. Fetch the password with YOUR credentials (not Terraform's state) and connect with
psqlfrom your laptop.terminal$ terraform apply -var enable_probe=true -auto-approve >/dev/nullaws ssm start-session --target $(terraform output -raw probe_instance_id) \--document-name AWS-StartPortForwardingSessionToRemoteHost \--parameters "host=$(terraform output -raw db_endpoint),portNumber=5432,localPortNumber=15432" &export PGPASSWORD=$(aws secretsmanager get-secret-value --secret-id "$(terraform output -raw db_master_secret_arn)" --query SecretString --output text | jq -r .password)psql "host=localhost port=15432 dbname=shoplite user=shoplite_admin sslmode=require" -c 'select version();'── expected output ──Starting session with SessionId: priya-0f1e2d...Port 15432 opened for sessionId priya-0f1e2d....version-------------------------------------------------------------------------------PostgreSQL 16.6 on aarch64-unknown-linux-gnu, compiled by gcc (GCC) 12.4.0, 64-bit(1 row)Turn the probe off again afterwards: `terraform apply -var enable_probe=false`.
Checkpoint — you should now have
- ✓An encrypted, non-public Postgres 16 instance runs in the database subnets.
- ✓Terraform code and state contain no password;
terraform output db_master_secret_arnshows only an ARN. - ✓You connected with
psqlthrough an SSM tunnel using the secret from Secrets Manager. - ✓In dev: deletion protection off and final snapshot skipped. In prod these flip automatically.
Part 4
Break it on purpose
Make each change, run the command, and read the error before revealing the diagnosis. Recognising these messages on sight is what makes you fast on a real team. Undo the change afterwards.
Break #1
Rename the database identifier
Change identifier to "${local.name_prefix}-postgres" and run terraform plan. Do NOT apply.
Break #2
Upgrade the major version by just changing the number
Set engine_version = "17" and apply.
Part 5
Interview questions from this mission
How do you create an RDS database with Terraform without the master password ending up in state?
Which RDS settings should differ between dev and prod, and how do you keep them from being mixed up?
Why set engine_version = "16" rather than "16.4"?
Before you stop