Mission 7.1 · Stage 7 — Team Workflow
Plan on Pull Request, Apply on Merge — with OIDC
Goal: Every pull request touching infra gets a plan for dev and prod posted as a comment; merging to main applies dev automatically and prod after an approval — and no AWS key exists anywhere.
By the end of this mission
- Let GitHub Actions assume AWS roles through OIDC with no stored secrets
- Separate a read-only plan role from a write apply role
- Post plans on pull requests and gate prod applies with GitHub Environments
- Prevent concurrent pipeline runs from fighting over state
Part 1
Understand it first
Why CI owns applies
When applies happen from CI, every change to real infrastructure has a pull request (the why), a reviewed plan (the what), an approver (the who), and a log (the when). Laptops stop needing write access to prod at all. That removes the single largest source of 'someone ran apply with uncommitted code' incidents.
OIDC: short-lived credentials, no secrets
GitHub can issue each workflow run a signed identity token that says, for example, 'this is repo you/shoplite, running for a pull request' or 'running in the prod environment'. AWS IAM trusts GitHub's OIDC provider, and a role's trust policy matches on that token's sub claim. The workflow exchanges the token for temporary credentials that last about an hour. No access key is ever stored in GitHub. This is the pattern from the AWS course, Topic 1.4, applied here.
Because the trust policy can match the context, you can give pull requests (untrusted code) only a READ-ONLY role that can plan, and allow the WRITE role only for runs in the protected prod environment on main.
Plan on PR, re-plan on merge
The PR plan is for REVIEW. By the time the PR merges, other PRs may have merged and state may have changed, so the saved PR plan could be stale. On merge, the pipeline plans again against the latest state and applies that exact saved plan in the same job. Serialising runs per environment with a concurrency group ensures two merges never apply at once; the state lock (Mission 1.2) is the last line of defence, not the first.
Part 2
Your project after this mission · 2 files change
- .github/
- workflows/
- terraform.ymlnew
- bootstrap/
- github-oidc.tfnew
- main.tf
- infra/
- envs/
- dev.s3.tfbackend
- dev.tfvars
- prod.s3.tfbackend
- prod.tfvars
- database.tf
- ecs.tf
- main.tf
- providers.tf
- security.tf
- tf
- modules/
- ecs-service/
- main.tf
- variables.tf
- network/
- main.tf
Part 3
Build it, step by step
- 1
Trust GitHub from AWS (in the bootstrap config, per account)
One OIDC provider per account, and two roles. The PLAN role trusts any pull request from your repo and gets read-only access plus state access (it needs to write the lock file). The DEPLOYER role, which you've been assuming yourself since Mission 6.3, now also trusts GitHub, but ONLY for runs in the
devorprodGitHub environment. Apply this with the bootstrap config in each account.bootstrap/github-oidc.tfwhole filehcl variable "github_repo" { type = string default = "you/shoplite" } variable "environment" { type = string # dev or prod — which account this bootstrap runs in } resource "aws_iam_openid_connect_provider" "github" { url = "https://token.actions.githubusercontent.com" client_id_list = ["sts.amazonaws.com"] } data "aws_iam_policy_document" "ci_plan_trust" { statement { actions = ["sts:AssumeRoleWithWebIdentity"] principals { type = "Federated" identifiers = [aws_iam_openid_connect_provider.github.arn] } condition { test = "StringEquals" variable = "token.actions.githubusercontent.com:aud" values = ["sts.amazonaws.com"] } condition { test = "StringEquals" variable = "token.actions.githubusercontent.com:sub" values = ["repo:${var.github_repo}:pull_request"] } } } resource "aws_iam_role" "ci_plan" { name = "shoplite-ci-plan" assume_role_policy = data.aws_iam_policy_document.ci_plan_trust.json } resource "aws_iam_role_policy_attachment" "ci_plan_readonly" { role = aws_iam_role.ci_plan.name policy_arn = "arn:aws:iam::aws:policy/ReadOnlyAccess" } data "aws_iam_policy_document" "ci_plan_state" { statement { sid = "StateReadAndLock" actions = ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket"] resources = [aws_s3_bucket.tfstate.arn, "${aws_s3_bucket.tfstate.arn}/*"] } statement { sid = "NeverReadSecretValues" effect = "Deny" actions = ["secretsmanager:GetSecretValue"] resources = ["*"] } } resource "aws_iam_role_policy" "ci_plan_state" { role = aws_iam_role.ci_plan.id policy = data.aws_iam_policy_document.ci_plan_state.json } # The deployer role (created during account setup) gets an extra trust statement: # sub = "repo:${var.github_repo}:environment:${var.environment}" # so only jobs running in the matching GitHub Environment can assume it. - 2
Create GitHub Environments
In the repo settings, create environments
devandprod. GiveprodREQUIRED REVIEWERS (you, or your team) and restrict it to themainbranch. Store each environment's account ID as an environment variable,AWS_ACCOUNT_ID, and also as repository variablesDEV_ACCOUNT_IDandPROD_ACCOUNT_IDfor the PR plan matrix. These aren't secrets; OIDC needs none. - 3
The workflow
Read it in three parts.
planruns for pull requests as a matrix over both environments with the read-only role, and posts each plan as a PR comment.apply-devruns on push to main, re-plans, and applies that saved plan.apply-prodneedsapply-devand runs in theprodenvironment, so it waits for approval.concurrencyserialises runs per environment and never cancels a running apply..github/workflows/terraform.ymlwhole fileyaml name: terraform on: pull_request: paths: ["infra/**", "modules/**"] push: branches: [main] paths: ["infra/**", "modules/**"] permissions: id-token: write # OIDC token for AWS contents: read pull-requests: write # plan comments env: TF_IN_AUTOMATION: "true" TF_INPUT: "false" AWS_REGION: ap-south-1 jobs: plan: if: github.event_name == 'pull_request' runs-on: ubuntu-latest strategy: fail-fast: false matrix: env: [dev, prod] concurrency: plan-${{ matrix.env }}-${{ github.head_ref }} steps: - uses: actions/checkout@v4 - uses: hashicorp/setup-terraform@v3 with: terraform_version: 1.13.3 terraform_wrapper: false - uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: arn:aws:iam::${{ vars[format('{0}_ACCOUNT_ID', matrix.env)] }}:role/shoplite-ci-plan aws-region: ${{ env.AWS_REGION }} - name: fmt run: terraform fmt -check -recursive - name: plan working-directory: infra run: | ./tf ${{ matrix.env }} validate ./tf ${{ matrix.env }} plan -lock-timeout=5m -no-color -out=tfplan terraform show -no-color tfplan > plan.txt - name: comment uses: actions/github-script@v7 with: script: | const plan = require('fs').readFileSync('infra/plan.txt', 'utf8'); const summary = plan.split('\n').find(l => l.startsWith('Plan:') || l.startsWith('No changes')) ?? ''; const body = `### \`${{ matrix.env }}\` — ${summary}\n<details><summary>Full plan</summary>\n\n\`\`\`\n${plan.slice(-60000)}\n\`\`\`\n</details>`; await github.rest.issues.createComment({ ...context.repo, issue_number: context.issue.number, body }); apply-dev: if: github.event_name == 'push' runs-on: ubuntu-latest environment: dev concurrency: group: apply-dev cancel-in-progress: false steps: &apply-steps - uses: actions/checkout@v4 - uses: hashicorp/setup-terraform@v3 with: terraform_version: 1.13.3 terraform_wrapper: false - uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/shoplite-deployer aws-region: ${{ env.AWS_REGION }} - name: plan and apply working-directory: infra run: | ./tf ${{ github.job == 'apply-prod' && 'prod' || 'dev' }} plan -lock-timeout=5m -out=tfplan terraform apply -lock-timeout=5m tfplan apply-prod: if: github.event_name == 'push' needs: apply-dev runs-on: ubuntu-latest environment: prod # required reviewers → the job waits for approval concurrency: group: apply-prod cancel-in-progress: false steps: *apply-steps - 4
Open a pull request and watch
Change something small: bump
api_memoryto 1024 inenvs/dev.tfvars. The PR gets two comments. Dev shows one task definition replacement and a service update; prod showsNo changes. Reviewers approve based on those comments, not on reading HCL diffs alone.terminal$ git switch -c api-memory && sed -i 's/^api_memory.*//' infra/envs/dev.tfvars && echo 'api_memory = 1024' >> infra/envs/dev.tfvarsgit commit -am 'dev: api memory 1024' && git push -u origin api-memory && gh pr create --fill── expected output ──https://github.com/you/shoplite/pull/42# PR comments posted by the workflow:### `dev` — Plan: 1 to add, 1 to change, 1 to destroy.### `prod` — No changes. Your infrastructure matches the configuration. - 5
Merge: dev applies, prod waits for you
After merging,
apply-devruns immediately.apply-prodshows 'Waiting for review' until a required reviewer approves in the Actions UI. Here prod has nothing to do, but the gate is always there. From now on, stop applying from your laptop.terminal$ gh pr merge 42 --squash && gh run watch── expected output ──✓ apply-dev Apply complete! Resources: 1 added, 1 changed, 1 destroyed.* apply-prod Waiting for review: prod needs approval to start deploying changes.
Checkpoint — you should now have
- ✓GitHub has no AWS access keys; both jobs authenticate through OIDC.
- ✓PRs get a plan comment per environment, produced with a read-only role.
- ✓Merges apply dev automatically; prod waits for a required reviewer.
- ✓Applies re-plan on main and apply that saved plan; runs are serialised per environment.
Part 4
Break it on purpose
Make each change, run the command, and read the error before revealing the diagnosis. Recognising these messages on sight is what makes you fast on a real team. Undo the change afterwards.
Break #1
Make the PR role able to apply
Temporarily point the plan job at the shoplite-deployer role instead of shoplite-ci-plan and push to the PR.
Break #2
Let a second run cancel an apply
Set cancel-in-progress: true on apply-dev, then merge two PRs a few seconds apart.
Part 5
Interview questions from this mission
How would you run Terraform in CI/CD on AWS securely?
Why re-plan on merge instead of applying the plan created for the pull request?
Why should PR jobs never have write credentials?