Mission 4.3 · Stage 4 — Data & Secrets
Guardrails: prevent_destroy, Preconditions, and check Blocks
Goal: Terraform refuses any plan that would destroy the ShopLite database, refuses to deploy prod without Multi-AZ, and warns after every apply if the live API isn't healthy.
By the end of this mission
- Use all four
lifecyclearguments and know their limits - Validate assumptions at plan time with preconditions and postconditions
- Add continuous health assertions with
checkblocks - Recreate a broken resource on purpose with
-replace
Part 1
Understand it first
The lifecycle meta-arguments
create_before_destroy (you've used it for security groups, target groups, and parameter groups) changes replacement order. ignore_changes (Mission 3.4) hands attributes to other controllers. prevent_destroy = true makes any plan that would destroy the resource an ERROR, whether from terraform destroy, a replacement, or deleted code. replace_triggered_by forces replacement when another resource or attribute changes.
Important limit: lifecycle arguments are processed before variables are evaluated, so they must be LITERALS. prevent_destroy = var.environment == "prod" is not allowed. For per-environment protection, combine a literal prevent_destroy with AWS-side protections like deletion_protection that can be variables.
Preconditions and postconditions
Variable validation checks a single input in isolation. PRECONDITIONS (in a resource's lifecycle) check assumptions across several values before the resource is planned, for example 'if environment is prod, multi_az must be true'. POSTCONDITIONS check what came back, for example 'the AMI this data source found really is arm64'. A failure stops the plan with your error message, pointing at the exact condition.
They turn tribal knowledge ('never deploy prod single-AZ') into rules the tool enforces on every run, for every person.
check blocks: assertions that warn
A check block (Terraform 1.5+) runs assertions at the end of every plan and apply, optionally with its own scoped data source. Unlike preconditions, a failing check produces a WARNING and doesn't block anything. That's right for things outside Terraform's control that you still want to know about, such as 'the public API returns 200'. After an apply that technically succeeded, a failed check tells you the service isn't actually healthy.
Part 2
Your project after this mission · 4 files change
- infra/
- alb.tf
- autoscaling.tf
- backend.tf
- checks.tfnew
- database.tfmodified
- ecr.tf
- ecs.tf
- iam.tf
- locals.tf
- logs.tf
- network.tf
- outputs.tf
- probe.tfmodified
- providers.tf
- refactors.tf
- security.tf
- storage.tf
- terraform.tfvars
- variables.tf
- versions.tfmodified
Part 3
Build it, step by step
- 1
Protect the database from Terraform itself
Add
prevent_destroyto the DB instance's lifecycle. It has to be a literal, so this applies to dev too. That's deliberate friction: to tear down dev you'll consciously remove the line first (see Clean up), and that removal shows up in review.infra/database.tfadd to filehcl Add inside resource "aws_db_instance" "main".
lifecycle { prevent_destroy = true precondition { condition = var.environment != "prod" || var.db_multi_az error_message = "Production databases must be Multi-AZ (set db_multi_az = true)." } precondition { condition = var.environment != "prod" || !startswith(var.db_instance_class, "db.t") error_message = "Burstable (db.t*) instance classes aren't allowed in prod." } } - 2
Verify an assumption about a data source
The probe's AMI lookup filters by name. If Amazon ever changed naming and the filter matched an x86 image, an ARM
t4ginstance would fail to launch with a confusing error. A postcondition turns that into a clear message at plan time.infra/probe.tfadd to filehcl Add inside data "aws_ami" "al2023_arm".
lifecycle { postcondition { condition = self.architecture == "arm64" error_message = "AMI lookup returned ${self.id} (${self.architecture}); expected an arm64 image for t4g instances." } } - 3
Add a live health check
The
httpprovider can make requests from within a check block. Addhttp = { source = "hashicorp/http", version = "~> 3.4" }torequired_providersand runterraform init. The check calls the database health endpoint through the public URL, which covers the ALB, ECS, the app, secrets, and RDS in one request.infra/checks.tfwhole filehcl check "api_health" { data "http" "db_health" { url = "http://${aws_lb.main.dns_name}/healthz/db" retry { attempts = 3 } } assert { condition = data.http.db_health.status_code == 200 error_message = "ShopLite /healthz/db returned ${data.http.db_health.status_code}: ${data.http.db_health.response_body}" } }terminal$ terraform init -upgrade >/dev/null && terraform apply── expected output ──...Plan: 0 to add, 0 to change, 0 to destroy....Apply complete! Resources: 0 added, 0 changed, 0 destroyed.A passing check prints nothing. You'll see it fail in Break it. - 4
Rebuild something on purpose with -replace
Occasionally a resource is broken in a way Terraform can't see, such as a corrupted instance or a stuck task definition.
-replace=ADDRESSplans a replacement of exactly that resource. It's the modern substitute for the oldterraform taint. Try it on the probe; it's safe.terminal$ terraform apply -var enable_probe=true -auto-approve >/dev/nullterraform plan -var enable_probe=true -replace='aws_instance.probe[0]'── expected output ──# aws_instance.probe[0] will be replaced, as requested-/+ resource "aws_instance" "probe" {...Plan: 1 to add, 0 to change, 1 to destroy.
Checkpoint — you should now have
- ✓The DB instance has
prevent_destroy = trueand two prod-only preconditions. - ✓The AMI data source has an
arm64postcondition. - ✓
check "api_health"runs on every plan/apply (silent when healthy). - ✓You know
-replacereplaces the deprecatedterraform taint.
Part 4
Break it on purpose
Make each change, run the command, and read the error before revealing the diagnosis. Recognising these messages on sight is what makes you fast on a real team. Undo the change afterwards.
Break #1
Try to replace the protected database
Repeat the rename from Mission 4.1 (identifier = "${local.name_prefix}-postgres") and run terraform plan.
Break #2
Make prevent_destroy depend on the environment
Change it to prevent_destroy = var.environment == "prod" and run terraform validate.
Break #3
Watch a check fail
Break the app's database access in a way the deploy won't notice: remove the DB_HOST entry from the container's environment list and apply.
Part 5
Interview questions from this mission
What are the lifecycle meta-arguments and a use case for each?
What's the difference between variable validation, preconditions, and check blocks?
How do you force Terraform to recreate a resource that looks fine in state?
Before you stop