Command Palette

Search for a command to run...

Hectal

Mission 3.3 · Stage 3 — Compute

ECS Fargate: Roles, Task Definition, Service

Goal: ShopLite 1.0.0 running as two Fargate tasks in private subnets, registered behind the ALB, logging to the imported log group, and answering on the public URL.

50 min ~$0.02/hour for two 0.25 vCPU / 0.5 GB ARM tasks 8 steps 3 break-it drills

By the end of this mission

  • Create separate execution and task roles and explain the difference
  • Write a task definition with jsonencode, references, and tostring
  • Wire a service to subnets, security groups, and the target group
  • Use wait_for_steady_state so an apply only succeeds when the app is healthy

Part 1

Understand it first

Two roles, two jobs

The EXECUTION ROLE is used by ECS itself, before and around your container: pulling the image from ECR, fetching secrets, and writing logs. The TASK ROLE is used by YOUR code for its own AWS calls (S3 uploads in Stage 4). They're separate so that application code never gets the platform's permissions, and the platform never gets the application's. The same split is covered in the AWS course, Topic 5.1.

Right now the task role only allows ECS Exec (so you can open a shell in a task). Stage 4 gives it S3 access, scoped to one bucket.

Task definitions are immutable revisions

Every change to a task definition (new image tag, new env var, more memory) creates a NEW revision (shoplite-dev-api:1, :2, ...). In Terraform that shows as the task definition being REPLACED and the service being UPDATED IN PLACE to point at the new revision. That's normal, and it's how every deploy looks from Mission 3.4 onwards. Old revisions stay registered (marked inactive) so you can roll back.

Applies that wait for health

By default Terraform considers the service 'created' as soon as the API call succeeds, even if every task then crashes. wait_for_steady_state = true makes the apply wait until the service has the desired number of healthy, running tasks, so a green apply means a working app. Combined with the deployment circuit breaker (automatic rollback on repeated failures), a bad deploy fails the apply loudly instead of quietly breaking production.

A real depends_on

An ECS service can only register tasks with a target group that's already attached to a load balancer through a listener. The service references the target group, but nothing references the listener, so Terraform might create the service first and ECS would reject it. depends_on = [aws_lb_listener.http] expresses that ordering, like the NAT/IGW case in Mission 2.2.

Request path after this missiondiagram
Rendering diagram…

Part 2

Your project after this mission · 4 files change

shoplite/
  • infra/
    • alb.tf
    • backend.tf
    • ecr.tf
    • ecs.tfnew
    • iam.tfnew
    • locals.tf
    • logs.tf
    • network.tf
    • outputs.tfmodified
    • probe.tf
    • providers.tf
    • refactors.tf
    • security.tf
    • storage.tf
    • terraform.tfvars
    • variables.tfmodified
    • versions.tf

Part 3

Build it, step by step

  1. 1

    Make sure NAT is on

    Tasks in private subnets pull their image from ECR over the internet, through NAT. If you switched NAT off at the end of Stage 2, set enable_nat_gateway = true again. VPC interface endpoints for ECR would avoid this, as in the AWS course, Topic 3.3.

  2. 2

    Deploy variables

    image_tag is the variable that deploys change. The size variables use the smallest Fargate size, which is plenty for ShopLite.

    infra/variables.tfadd to filehcl
    variable "image_tag" {
      description = "ShopLite API image tag to run."
      type        = string
      default     = "1.0.0"
    }
    
    variable "api_cpu" {
      type    = number
      default = 256 # 0.25 vCPU
    }
    
    variable "api_memory" {
      type    = number
      default = 512 # MiB
    }
    
    variable "api_desired_count" {
      type    = number
      default = 2
    }
  3. 3

    Execution role and task role

    Both trust ecs-tasks.amazonaws.com. The execution role gets the AWS-managed policy for pulling from ECR and writing logs. The task role gets only the four ssmmessages actions ECS Exec needs.

    infra/iam.tfwhole filehcl
    data "aws_iam_policy_document" "ecs_tasks_assume" {
      statement {
        actions = ["sts:AssumeRole"]
        principals {
          type        = "Service"
          identifiers = ["ecs-tasks.amazonaws.com"]
        }
      }
    }
    
    # Used by ECS: pull image, write logs (and fetch secrets in Stage 4).
    resource "aws_iam_role" "ecs_execution" {
      name               = "${local.name_prefix}-ecs-execution"
      assume_role_policy = data.aws_iam_policy_document.ecs_tasks_assume.json
    }
    
    resource "aws_iam_role_policy_attachment" "ecs_execution" {
      role       = aws_iam_role.ecs_execution.name
      policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
    }
    
    # Used by ShopLite's own code.
    resource "aws_iam_role" "api_task" {
      name               = "${local.name_prefix}-api-task"
      assume_role_policy = data.aws_iam_policy_document.ecs_tasks_assume.json
    }
    
    data "aws_iam_policy_document" "api_task" {
      statement {
        sid = "EcsExec"
        actions = [
          "ssmmessages:CreateControlChannel",
          "ssmmessages:CreateDataChannel",
          "ssmmessages:OpenControlChannel",
          "ssmmessages:OpenDataChannel",
        ]
        resources = ["*"]
      }
    }
    
    resource "aws_iam_role_policy" "api_task" {
      name   = "api-task"
      role   = aws_iam_role.api_task.id
      policy = data.aws_iam_policy_document.api_task.json
    }
  4. 4

    Cluster and task definition

    The container definition is ordinary HCL inside jsonencode, with references to the ECR URL, the log group from Mission 1.3, and variables. Environment variable values must be strings, hence tostring(var.app_port). runtime_platform selects ARM64 to match the image.

    infra/ecs.tfwhole filehcl
    resource "aws_ecs_cluster" "main" {
      name = local.name_prefix
    
      setting {
        name  = "containerInsights"
        value = "enabled"
      }
    }
    
    resource "aws_ecs_task_definition" "api" {
      family                   = "${local.name_prefix}-api"
      requires_compatibilities = ["FARGATE"]
      network_mode             = "awsvpc"
      cpu                      = var.api_cpu
      memory                   = var.api_memory
      execution_role_arn       = aws_iam_role.ecs_execution.arn
      task_role_arn            = aws_iam_role.api_task.arn
    
      runtime_platform {
        operating_system_family = "LINUX"
        cpu_architecture        = "ARM64"
      }
    
      container_definitions = jsonencode([{
        name      = "api"
        image     = "${aws_ecr_repository.api.repository_url}:${var.image_tag}"
        essential = true
    
        portMappings = [{ containerPort = var.app_port, protocol = "tcp" }]
    
        environment = [
          { name = "PORT", value = tostring(var.app_port) },
          { name = "APP_VERSION", value = var.image_tag },
        ]
    
        logConfiguration = {
          logDriver = "awslogs"
          options = {
            "awslogs-group"         = aws_cloudwatch_log_group.api.name
            "awslogs-region"        = var.aws_region
            "awslogs-stream-prefix" = "api"
          }
        }
      }])
    }
  5. 5

    The service

    Private subnets, the app security group, no public IP. health_check_grace_period_seconds gives a new task 30 seconds before failed ALB checks count against it. The circuit breaker settings with minimum 100% / maximum 200% mean a deploy starts new tasks alongside the old ones and only stops old tasks once new ones are healthy, giving zero-downtime rolling deploys.

    infra/ecs.tfadd to filehcl
    resource "aws_ecs_service" "api" {
      name                              = "${local.name_prefix}-api"
      cluster                           = aws_ecs_cluster.main.id
      task_definition                   = aws_ecs_task_definition.api.arn
      desired_count                     = var.api_desired_count
      launch_type                       = "FARGATE"
      enable_execute_command            = true
      health_check_grace_period_seconds = 30
      wait_for_steady_state             = true
    
      deployment_minimum_healthy_percent = 100
      deployment_maximum_percent         = 200
    
      deployment_circuit_breaker {
        enable   = true
        rollback = true
      }
    
      network_configuration {
        subnets          = [for s in aws_subnet.private : s.id]
        security_groups  = [aws_security_group.app.id]
        assign_public_ip = false
      }
    
      load_balancer {
        target_group_arn = aws_lb_target_group.api.arn
        container_name   = "api"
        container_port   = var.app_port
      }
    
      # The target group must be attached to the ALB (via a listener) before ECS can use it.
      depends_on = [aws_lb_listener.http]
    }
  6. 6

    Apply and watch it wait

    The service shows Still creating... until both tasks pass the ALB health check, usually 1–3 minutes. Watch the ECS console's Events tab in parallel to see tasks start and register.

    terminal
    $ terraform apply
    ── expected output ──
    Plan: 8 to add, 0 to change, 0 to destroy.
    ...
    aws_ecs_service.api: Creating...
    aws_ecs_service.api: Still creating... [1m0s elapsed]
    aws_ecs_service.api: Still creating... [2m0s elapsed]
    aws_ecs_service.api: Creation complete after 2m14s [id=arn:aws:ecs:ap-south-1:123456789012:service/shoplite-dev/shoplite-dev-api]
     
    Apply complete! Resources: 8 added, 0 changed, 0 destroyed.
  7. 7

    Call ShopLite

    The same URL that returned 503 now reaches the app through the ALB. Run it a few times; requests are spread across both tasks.

    terminal
    $ URL=$(terraform output -raw api_url)
    curl -s $URL; echo
    curl -s $URL/products; echo
    ── expected output ──
    {"service":"shoplite-api","version":"1.0.0"}
    [{"id":1,"name":"Masala Chai","price":120},{"id":2,"name":"Filter Coffee","price":90},{"id":3,"name":"Mango Lassi","price":150}]
  8. 8

    Look inside: targets, logs, a shell

    Add an output for the target group so health checks are one command away. Outputs exist to save you exactly this kind of lookup. Both targets should be healthy, and the logs show the startup line from server.js. aws ecs execute-command then gives you a shell with no SSH, using the task role permissions you added.

    infra/outputs.tfadd to filehcl
    output "api_target_group_arn" {
      value = aws_lb_target_group.api.arn
    }
    terminal
    $ terraform apply -auto-approve >/dev/null
    aws elbv2 describe-target-health --target-group-arn $(terraform output -raw api_target_group_arn) \
    --query 'TargetHealthDescriptions[].[Target.Id,TargetHealth.State]' --output text
    aws logs tail /shoplite/dev/api --since 10m
    ── expected output ──
    10.20.10.87 healthy
    10.20.11.142 healthy
    2026-09-26T12:40:03 api/api/5e1c... shoplite-api 1.0.0 listening on 8080
    2026-09-26T12:40:05 api/api/9ab2... shoplite-api 1.0.0 listening on 8080

Checkpoint — you should now have

  • ✓curl $(terraform output -raw api_url) returns {"service":"shoplite-api","version":"1.0.0"}.
  • ✓Two tasks run in private subnets with no public IPs; both targets are healthy.
  • ✓Logs appear in /shoplite/dev/api (the log group you imported in 1.3).
  • ✓The execution role and task role are separate, and the task role has only ECS Exec permissions.

Part 4

Break it on purpose

Make each change, run the command, and read the error before revealing the diagnosis. Recognising these messages on sight is what makes you fast on a real team. Undo the change afterwards.

Break #1

Use an instance target group

Change the target group's target_type to "instance" and apply. (Because of create_before_destroy, a new target group is created before the service is updated.)

terminal
$ terraform apply
── what you'll see ──
aws_ecs_service.api: Modifying... [id=arn:aws:ecs:...:service/shoplite-dev/shoplite-dev-api]
╷
│ Error: updating ECS Service (arn:aws:ecs:...:service/shoplite-dev/shoplite-dev-api):
│ operation error ECS: UpdateService, https response error StatusCode: 400,
│ InvalidParameterException: The provided target group
│ arn:aws:elasticloadbalancing:...:targetgroup/api-2026.../... has target type
│ instance, which is incompatible with the awsvpc network mode specified in
│ the task definition.
╵

Break #2

Point the health check at the wrong path

Change the health check path to /health (no z) and apply. Then check the service events.

terminal
$ aws ecs describe-services --cluster shoplite-dev --services shoplite-dev-api --query 'services[0].events[:3].message' --output text
── what you'll see ──
(service shoplite-dev-api) (port 8080) is unhealthy in target-group api-2026... due to (reason Health checks failed with these codes: [404]).
(service shoplite-dev-api) has stopped 1 running tasks: (task 5e1c...).
(service shoplite-dev-api) has started 1 tasks: (task 7d0f...).

Break #3

Deploy an image tag that doesn't exist

Run terraform apply -var image_tag=9.9.9.

terminal
$ terraform apply -var image_tag=9.9.9
── what you'll see ──
# aws_ecs_task_definition.api must be replaced
-/+ resource "aws_ecs_task_definition" "api" {
~ container_definitions = jsonencode(
~ [ { ~ image = "...shoplite/api:1.0.0" -> "...shoplite/api:9.9.9" ... } ]
) # forces replacement
...
aws_ecs_service.api: Still modifying... [5m0s elapsed]
╷
│ Error: waiting for ECS Service (...shoplite-dev-api) update: deployment
│ failed: ECS deployment circuit breaker: tasks failed to start.
╵

Part 5

Interview questions from this mission

01

What's the difference between an ECS task execution role and a task role?

02

Why does changing an image tag replace the task definition instead of updating it?

03

What does wait_for_steady_state add, and what's the trade-off?

04

An ECS service keeps starting and stopping tasks. How do you debug it?

0/4 · 0%