Command Palette

Search for a command to run...

Hectal

Mission 1.2 · Stage 1 — State

Move State to S3 with Locking

Goal: A dedicated, versioned, encrypted state bucket built by its own bootstrap config, and ShopLite's state migrated into it with S3-native locking.

40 min Cents per month — S3 storage and requests for a few KB of state 7 steps 3 break-it drills

By the end of this mission

  • Solve the bootstrap problem: where the state bucket's own state lives
  • Configure the S3 backend with encryption and use_lockfile
  • Migrate existing local state with terraform init -migrate-state
  • Recover an older state version from S3 versioning

Part 1

Understand it first

What a backend is

A BACKEND decides where state is stored and how it's locked. The default local backend writes terraform.tfstate next to your code. Remote backends (S3, Azure Blob, GCS, HCP Terraform, Postgres, and others) store it centrally, so everyone and every pipeline reads the same state and shares one lock.

For AWS teams the S3 backend is the standard choice. Since Terraform 1.10 it can lock using S3 itself (use_lockfile = true, which creates a .tflock object with a conditional write), so the separate DynamoDB lock table older guides require is no longer needed.

The chicken-and-egg: bootstrapping the state bucket

The state bucket must exist before any configuration can store state in it. So who creates the bucket, and where does THAT state live? The usual answer is a tiny, separate BOOTSTRAP configuration that creates only the state bucket, applied once with local state. Its state is then migrated into the bucket it just created (under its own key), so nothing is left on a laptop.

Keep the bootstrap config separate from application infrastructure. It changes almost never, and you don't want ShopLite's everyday applies to have any path to deleting the bucket that holds their own state.

What makes a state bucket production-grade

VERSIONING: every state write keeps the previous version, so a corrupted or wrong write can be rolled back. Of all the settings here, this one matters most. ENCRYPTION: state holds secrets, so encrypt it (SSE-S3 by default, or SSE-KMS with a customer-managed key if you want key-level access control and audit). BLOCK PUBLIC ACCESS: obviously. prevent_destroy: Terraform refuses any plan that would delete the bucket. A narrow IAM policy: only Terraform roles can read or write it.

Use one bucket for all of a team's state, with a separate KEY (path) per configuration and environment: shoplite/dev/terraform.tfstate, shoplite/prod/terraform.tfstate. Stage 6 builds on this.

Backend blocks can't use variables

The backend is configured during init, before variables are evaluated, so the backend block only accepts literal values. When you need different values per environment, pass them at init time with -backend-config=..., or keep one small backend.tf per environment directory (Stage 6).

Two configurations, one bucketdiagram
Rendering diagram…

Part 2

Your project after this mission · 4 files change

shoplite/
  • bootstrap/
    • .terraform.lock.hclnew
    • backend.tfnew
    • main.tfnew
  • infra/
    • .terraform.lock.hcl
    • backend.tfnew
    • locals.tf
    • main.tf
    • outputs.tf
    • providers.tf
    • terraform.tfvars
    • variables.tf
    • versions.tf
  • .gitignore

Part 3

Build it, step by step

  1. 1

    Write the bootstrap configuration

    A separate folder with its own providers. prevent_destroy means even terraform destroy in this folder fails, which you want for the bucket holding every state file you own. noncurrent_version_expiration keeps 90 days of old state versions without growing forever.

    bootstrap/main.tfwhole filehcl
    terraform {
      required_version = ">= 1.10"
      required_providers {
        aws    = { source = "hashicorp/aws", version = "~> 6.0" }
        random = { source = "hashicorp/random", version = "~> 3.6" }
      }
    }
    
    provider "aws" {
      region = "ap-south-1"
      default_tags {
        tags = { Project = "shoplite", ManagedBy = "terraform", Stack = "bootstrap" }
      }
    }
    
    resource "random_id" "suffix" {
      byte_length = 2
    }
    
    resource "aws_s3_bucket" "tfstate" {
      bucket = "shoplite-tfstate-${random_id.suffix.hex}"
    
      lifecycle {
        prevent_destroy = true
      }
    }
    
    resource "aws_s3_bucket_versioning" "tfstate" {
      bucket = aws_s3_bucket.tfstate.id
      versioning_configuration {
        status = "Enabled"
      }
    }
    
    resource "aws_s3_bucket_server_side_encryption_configuration" "tfstate" {
      bucket = aws_s3_bucket.tfstate.id
      rule {
        apply_server_side_encryption_by_default {
          sse_algorithm = "aws:kms"
        }
        bucket_key_enabled = true
      }
    }
    
    resource "aws_s3_bucket_public_access_block" "tfstate" {
      bucket                  = aws_s3_bucket.tfstate.id
      block_public_acls       = true
      block_public_policy     = true
      ignore_public_acls      = true
      restrict_public_buckets = true
    }
    
    resource "aws_s3_bucket_lifecycle_configuration" "tfstate" {
      bucket = aws_s3_bucket.tfstate.id
      rule {
        id     = "expire-old-state-versions"
        status = "Enabled"
        filter {}
        noncurrent_version_expiration {
          noncurrent_days = 90
        }
      }
    }
    
    output "state_bucket" {
      value = aws_s3_bucket.tfstate.bucket
    }
  2. 2

    Apply the bootstrap with local state

    This is the only apply in the course that deliberately uses local state. Note the bucket name from the output; you'll use it in two backend blocks.

    terminal
    $ cd ../bootstrap
    terraform init && terraform apply
    ── expected output ──
    ...
    Plan: 6 to add, 0 to change, 0 to destroy.
    ...
    Apply complete! Resources: 6 added, 0 changed, 0 destroyed.
     
    Outputs:
     
    state_bucket = "shoplite-tfstate-c41e"
  3. 3

    Move the bootstrap's own state into the bucket

    Add a backend block, using the literal bucket name because backend blocks can't use variables, and re-run init. Terraform detects the backend change and offers to copy the existing local state. Answer yes. Afterwards the local terraform.tfstate is empty and can be deleted.

    bootstrap/backend.tfwhole filehcl
    terraform {
      backend "s3" {
        bucket       = "shoplite-tfstate-c41e"
        key          = "bootstrap/terraform.tfstate"
        region       = "ap-south-1"
        encrypt      = true
        use_lockfile = true
      }
    }
    terminal
    $ terraform init -migrate-state
    ── expected output ──
    Initializing the backend...
    Do you want to copy existing state to the new backend?
    Pre-existing state was found while migrating the previous "local" backend to the
    newly configured "s3" backend. No existing state was found in the newly
    configured "s3" backend. Do you want to copy this state to the new "s3"
    backend? Enter "yes" to copy and "no" to start with an empty state.
     
    Enter a value: yes
     
    Successfully configured the backend "s3"! Terraform will automatically
    use this backend unless the backend configuration changes.
  4. 4

    Point ShopLite at the bucket and migrate

    Same pattern for the main config, with a different key. The key encodes project and environment, a convention Stage 6 relies on.

    infra/backend.tfwhole filehcl
    terraform {
      backend "s3" {
        bucket       = "shoplite-tfstate-c41e"
        key          = "shoplite/dev/terraform.tfstate"
        region       = "ap-south-1"
        encrypt      = true
        use_lockfile = true
      }
    }
    terminal
    $ cd ../infra
    terraform init -migrate-state
    terraform plan
    ── expected output ──
    Successfully configured the backend "s3"! Terraform will automatically
    use this backend unless the backend configuration changes.
    ...
    No changes. Your infrastructure matches the configuration.
    `No changes` proves the migrated state is complete and correct.
  5. 5

    Delete the local copies and check the bucket

    Local state files are now stale duplicates, and keeping them only invites someone to use them by accident. Remove them, then look at what's in S3: one object per configuration.

    terminal
    $ rm -f terraform.tfstate terraform.tfstate.backup ../bootstrap/terraform.tfstate*
    aws s3 ls s3://shoplite-tfstate-c41e --recursive
    ── expected output ──
    2026-09-26 11:02:17 9214 bootstrap/terraform.tfstate
    2026-09-26 11:05:41 7688 shoplite/dev/terraform.tfstate
  6. 6

    See the lock object appear during an operation

    Start terraform apply and leave it at the prompt. In another terminal, list the bucket again. A .tflock object exists for as long as the operation runs; S3's conditional writes guarantee only one process can create it. Answer no and it disappears.

    terminal
    $ aws s3 ls s3://shoplite-tfstate-c41e/shoplite/dev/
    ── expected output ──
    2026-09-26 11:05:41 7688 terraform.tfstate
    2026-09-26 11:09:02 233 terraform.tfstate.tflock
  7. 7

    Know how to roll state back

    With versioning, every state write is recoverable. List versions, download an older one, inspect it, and, only if you're sure, push it back with terraform state push. You'll rarely need this, but it's worth knowing before an incident rather than during one.

    terminal
    $ aws s3api list-object-versions --bucket shoplite-tfstate-c41e \
    --prefix shoplite/dev/terraform.tfstate --query 'Versions[].[VersionId,LastModified]' --output table
    ── expected output ──
    -------------------------------------------------------------
    | ListObjectVersions |
    +-----------------------------------+-----------------------+
    | 3HL4kqtJlcpXroDTDmJ.rmSpXd3dIbrHY| 2026-09-26T11:05:41Z |
    | 0pmRZ.S2hJ8qOy_.tW7YI7d9hxHO2Lp4 | 2026-09-26T11:05:40Z |
    +-----------------------------------+-----------------------+

Checkpoint — you should now have

  • ✓The state bucket has versioning, KMS encryption, public access blocked, and prevent_destroy.
  • ✓Both bootstrap/ and infra/ store state in S3 under different keys; no local .tfstate files remain.
  • ✓terraform plan in infra/ shows no changes after migration.
  • ✓You've seen the .tflock object appear during an apply and disappear afterwards.
  • ✓backend.tf files are committed (they contain no secrets).

Part 4

Break it on purpose

Make each change, run the command, and read the error before revealing the diagnosis. Recognising these messages on sight is what makes you fast on a real team. Undo the change afterwards.

Break #1

Two people apply against the remote state

Simulate a teammate: in terminal 1 run terraform apply and leave the prompt open; in terminal 2 run terraform apply.

terminal
$ terraform apply
── what you'll see ──
╷
│ Error: Error acquiring the state lock
│
│ Error message: operation error S3: PutObject, https response error
│ StatusCode: 412, RequestID: 5FQ..., api error PreconditionFailed: At least
│ one of the pre-conditions you specified did not hold
│ Lock Info:
│ ID: a1c9e0b2-6d3f-4e8a-9b17-2c5d8e0f4a63
│ Path: shoplite-tfstate-c41e/shoplite/dev/terraform.tfstate
│ Operation: OperationTypeApply
│ Who: priya@laptop
│ Version: 1.13.3
│ Created: 2026-09-26 11:12:40.5521 +0000 UTC
│ Info:
╵

Break #2

Use a variable in the backend block

In infra/backend.tf, change the key to key = "shoplite/${var.environment}/terraform.tfstate" and run terraform init.

terminal
$ terraform init
── what you'll see ──
Initializing the backend...
╷
│ Error: Variables not allowed
│
│ on backend.tf line 4, in terraform:
│ 4: key = "shoplite/${var.environment}/terraform.tfstate"
│
│ Variables may not be used here.
╵

Break #3

Change the backend without re-initialising

Change key to shoplite/dev2/terraform.tfstate in infra/backend.tf and run terraform plan without init.

terminal
$ terraform plan
── what you'll see ──
╷
│ Error: Backend initialization required, please run "terraform init"
│
│ Reason: Backend configuration block has changed
│
│ The "backend" is the interface that Terraform uses to store state,
│ perform operations, etc. If this message is showing up, it means that the
│ Terraform configuration you're using is using a custom configuration for
│ the Terraform backend.
╵

Part 5

Interview questions from this mission

01

How would you set up Terraform state for a team on AWS?

02

How does S3-native state locking work?

03

Where does the state for the state bucket itself live?

04

What's the difference between terraform init -migrate-state and -reconfigure?

0/4 · 0%