Amazon ECS Deployment Strategies and Auto Scaling

By Oleksandr Andrushchenko — Published on

Amazon ECS Deployment Strategies and Auto Scaling
Amazon ECS Deployment Strategies and Auto Scaling

Amazon ECS deployment strategies and auto scaling determine how containerized services change over time. Deployments control how new versions replace old versions, while auto scaling controls how many tasks run as traffic, resource usage, or queue backlog changes.

A production ECS service should handle both safely. A good deployment strategy prevents broken releases from taking down the service, and a good scaling strategy adds capacity before users or background jobs experience serious delay.

Table of Contents

Deployment and Scaling Model

ECS deployments and auto scaling both change the number or version of running tasks. A deployment replaces old task definition revisions with new ones. Auto scaling changes desired task count based on workload signals.

These two systems must work together. A service can be healthy during normal traffic but fail during deployment if there is not enough temporary capacity. A service can also deploy correctly but become overloaded if scaling rules react too slowly.

ECS Service
  Desired Count = 4

Deployment:
  Replace revision 12 tasks with revision 13 tasks

Auto Scaling:
  Increase desired count from 4 to 8 when traffic grows
Concern Deployment Strategy Auto Scaling
Main purpose Change application version safely Change task count safely
Primary trigger New task definition revision CloudWatch metric or schedule
Main risk Bad release Too little or too much capacity
Success signal New tasks become healthy Metric returns to target range

ECS Deployment Strategies

ECS services can be deployed in several ways. The right strategy depends on failure tolerance, release risk, rollback requirements, traffic volume, and operational maturity.

The most common deployment strategy is rolling deployment. More controlled systems may use blue/green or canary release patterns.

Rolling Deployments

A rolling deployment gradually replaces old tasks with new tasks. ECS starts tasks from the new task definition revision, waits for them to become healthy, and stops old tasks.

Before deployment:
  Task A: revision 12
  Task B: revision 12
  Task C: revision 12
  Task D: revision 12

During deployment:
  Task A: revision 12
  Task B: revision 12
  Task C: revision 13
  Task D: revision 13

After deployment:
  Task A: revision 13
  Task B: revision 13
  Task C: revision 13
  Task D: revision 13

Rule of Thumb: rolling deployments are the best default for most ECS services when the application supports backward-compatible changes and fast health checks.

Blue/Green Deployments

A blue/green deployment runs the old version and the new version as separate environments or target groups. Traffic is shifted from the old version to the new version after validation.

Blue environment:
  Current production tasks

Green environment:
  New version tasks

Traffic shift:
  ALB / deployment controller moves traffic from blue to green

Blue/green deployments are useful when releases need stronger isolation and faster rollback. The trade-off is more infrastructure and more deployment complexity.

Strategy Rollback Speed Operational Complexity
Rolling Medium Lower
Blue/Green Fast Higher
Canary Fast for small traffic slice Higher

Canary Deployments

A canary deployment sends a small percentage of traffic to the new version before a full rollout. This reduces blast radius when a release has unknown production behavior.

Initial:
  95% traffic -> old version
   5% traffic -> new version

If metrics are healthy:
  50% traffic -> old version
  50% traffic -> new version

Final:
  100% traffic -> new version

Canary deployments are valuable for high-traffic systems, risky changes, and user-facing services with strong observability. They require good metrics because the decision to continue or rollback depends on real signals.

Deployment Configuration

ECS rolling deployments are controlled by service deployment settings. These settings decide how many old tasks can stop and how many new tasks can temporarily run during a release.

Bad deployment configuration can cause avoidable downtime even when the application code is correct.

Minimum Healthy Percent

Minimum healthy percent controls the lowest number of healthy tasks that must remain running during a deployment.

Desired Count = 4
Minimum Healthy Percent = 50%

Minimum healthy tasks during deployment:
  4 * 50% = 2 tasks

A lower value allows ECS to stop more old tasks before starting replacements. A higher value protects availability but may require more temporary capacity.

Maximum Percent

Maximum percent controls how many tasks can run temporarily during deployment compared with desired count.

Desired Count = 4
Maximum Percent = 200%

Maximum tasks during deployment:
  4 * 200% = 8 tasks

Production Note: maximum percent is especially important for zero-downtime rolling deployments. It allows ECS to start new tasks before stopping old tasks.

Deployment Circuit Breaker

The deployment circuit breaker can mark a deployment as failed when new tasks cannot reach a healthy steady state. With rollback enabled, ECS can automatically return to the previous working task definition.

Setting Purpose Production Value
Enable circuit breaker Detect failed deployments Prevents endless unhealthy rollout attempts.
Enable rollback Return to previous revision Reduces manual recovery time.

Important: automatic rollback depends on accurate health signals. If health checks are too weak, broken tasks may be considered healthy.

ECS Auto Scaling

ECS auto scaling changes the desired count of a service. It does not make individual tasks larger; it adds or removes task copies.

The correct scaling policy depends on workload type. APIs usually scale on request pressure or resource utilization. Workers usually scale on queue backlog or message age.

Service Auto Scaling

Service Auto Scaling integrates ECS services with Application Auto Scaling. The scalable dimension is the ECS service desired count.

Metric increases
  -> Scaling policy triggers
  -> ECS desired count increases
  -> ECS starts more tasks
  -> Load spreads across more capacity

Scaling has limits. Every service should define minimum capacity and maximum capacity to avoid both under-provisioning and runaway cost.

Target Tracking Scaling

Target tracking keeps a metric near a target value. For example, keep average CPU around 60% or ALB request count per target around a defined threshold.

Target Metric Good Fit
CPU utilization CPU-bound APIs and processors.
Memory utilization Memory-sensitive services.
ALB request count per target HTTP APIs with request-based load.

Target tracking is usually the simplest autoscaling strategy for ECS APIs because it handles scale-out and scale-in without manually defining every threshold.

Step Scaling

Step scaling changes capacity by different amounts based on how far a metric moves beyond a threshold.

Queue depth:
  100 messages  -> add 1 task
  500 messages  -> add 3 tasks
  1000 messages -> add 6 tasks

Step scaling is useful when workload pressure is not linear. Queue workers often benefit from step scaling because backlog growth may require aggressive capacity changes.

Scheduled Scaling

Scheduled scaling changes service capacity at known times. It is useful when traffic patterns are predictable.

Pattern Scheduled Scaling Use
Business-hours traffic Increase task count before users arrive.
Nightly batch processing Increase worker count before jobs start.
Known marketing campaign Pre-warm API capacity before launch.

Trade-off: scheduled scaling is reliable for predictable load, but it does not react to unexpected spikes by itself. It is often combined with target tracking or step scaling.

Choosing Scaling Metrics

Scaling should follow the real bottleneck. CPU, memory, request count, latency, and queue backlog all describe different kinds of pressure.

The best metric depends on whether the service is synchronous, asynchronous, CPU-bound, memory-bound, or dependency-bound.

API Scaling Metrics

APIs should scale to protect response time and availability. CPU is useful, but request count per target is often a better load signal for HTTP services.

Metric Use for Scaling? Use for Alerts?
ALB request count per target Yes Sometimes
CPU utilization Yes, for CPU-bound services Yes
Memory utilization Yes, for memory-heavy services Yes
Latency Carefully Yes
5xx error rate No Yes

Worker Scaling Metrics

Workers should scale to control backlog. CPU may be low even when a queue is behind, especially if workers spend time waiting on databases, APIs, or file systems.

Metric Meaning Production Use
Visible messages Queue backlog size Scale-out signal.
Oldest message age Processing delay Strong alert signal.
Processing duration Job cost Capacity planning.
DLQ count Repeated failures Critical alert.

Key Point: APIs scale to protect latency. Workers scale to protect backlog age.

Deployment and Scaling Interactions

Deployments and scaling can overlap. A service might be deploying a new version while traffic spikes or while a scaling policy changes desired count.

Production configuration should assume these events can happen at the same time.

Capacity During Deployments

Rolling deployments often need temporary extra capacity. If maximum percent is too low, ECS may need to stop old tasks before enough new tasks are ready.

Safer deployment:
  Desired Count = 4
  Minimum Healthy Percent = 100
  Maximum Percent = 200

Result:
  ECS can start new tasks before stopping old tasks.

This pattern reduces downtime risk but requires enough compute capacity. With Fargate, this usually means more temporary task capacity. With EC2 launch type, the cluster must have enough room to place additional tasks.

Scale-In During Release

Scale-in during deployment can remove capacity at the wrong time. Cooldowns and conservative scale-in rules help prevent unnecessary task termination during unstable periods.

Production Note: scale-out should usually be faster than scale-in. Adding capacity quickly protects availability, while removing capacity slowly prevents oscillation.

Ready-to-Use Example

This example shows an ECS Fargate service with rolling deployment settings, deployment circuit breaker, and target tracking auto scaling based on ALB request count per target.

The example assumes that the VPC, private subnets, ECS cluster, target group, security group, task execution role, and task role already exist.

CloudFormation Example

The CloudFormation snippet below defines the ECS task definition, service, scalable target, and scaling policy.

AWSTemplateFormatVersion: "2010-09-09"
Description: ECS service deployment strategy and auto scaling example

Parameters:
  ClusterName:
    Type: String
    Description: Existing ECS cluster name.

  ServiceName:
    Type: String
    Default: orders-api
    Description: ECS service name.

  ContainerImage:
    Type: String
    Description: Immutable container image URI, usually tagged with Git SHA.

  PrivateSubnetIds:
    Type: List
    Description: Private subnets where ECS tasks should run.

  TaskSecurityGroupId:
    Type: AWS::EC2::SecurityGroup::Id
    Description: Security group allowing inbound traffic from the ALB only.

  TargetGroupArn:
    Type: String
    Description: ALB target group ARN used by the ECS service.

  TaskExecutionRoleArn:
    Type: String
    Description: Role used by ECS to pull images and write logs.

  TaskRoleArn:
    Type: String
    Description: Role used by application code inside the container.

Resources:
  ApiLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      # Keep logs long enough for debugging without retaining them forever.
      LogGroupName: !Sub "/ecs/${ServiceName}"
      RetentionInDays: 14

  ApiTaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      # Family groups task definition revisions under one application name.
      Family: !Ref ServiceName

      # awsvpc gives each task its own ENI and security group.
      NetworkMode: awsvpc

      # Fargate removes EC2 container instance management.
      RequiresCompatibilities:
        - FARGATE

      # Tune CPU and memory from load testing, not guesses.
      Cpu: "512"
      Memory: "1024"

      ExecutionRoleArn: !Ref TaskExecutionRoleArn
      TaskRoleArn: !Ref TaskRoleArn

      ContainerDefinitions:
        - Name: api
          Image: !Ref ContainerImage
          Essential: true

          # The ALB target group forwards traffic to this container port.
          PortMappings:
            - ContainerPort: 8000
              Protocol: tcp

          # Keep non-sensitive config here. Secrets should use Secrets Manager or SSM.
          Environment:
            - Name: APP_ENV
              Value: production

          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: !Ref ApiLogGroup
              awslogs-region: !Ref AWS::Region
              awslogs-stream-prefix: ecs

  ApiService:
    Type: AWS::ECS::Service
    Properties:
      ServiceName: !Ref ServiceName
      Cluster: !Ref ClusterName
      LaunchType: FARGATE
      TaskDefinition: !Ref ApiTaskDefinition

      # Start with at least two tasks for basic production redundancy.
      DesiredCount: 2

      # Give the application time to start before ALB health checks affect deployment.
      HealthCheckGracePeriodSeconds: 60

      DeploymentConfiguration:
        # Keep all desired tasks healthy during rolling deployments.
        MinimumHealthyPercent: 100

        # Allow ECS to temporarily double task count during deployment.
        MaximumPercent: 200

        DeploymentCircuitBreaker:
          # Detect deployments that cannot reach a healthy steady state.
          Enable: true

          # Automatically return to the previous working revision on deployment failure.
          Rollback: true

      NetworkConfiguration:
        AwsvpcConfiguration:
          # Keep tasks private. Public traffic should enter through the ALB.
          AssignPublicIp: DISABLED
          Subnets: !Ref PrivateSubnetIds
          SecurityGroups:
            - !Ref TaskSecurityGroupId

      LoadBalancers:
        - ContainerName: api
          ContainerPort: 8000
          TargetGroupArn: !Ref TargetGroupArn

  ApiScalableTarget:
    Type: AWS::ApplicationAutoScaling::ScalableTarget
    Properties:
      # ECS service desired count is the scalable dimension.
      ServiceNamespace: ecs
      ScalableDimension: ecs:service:DesiredCount

      # ResourceId format must be service/cluster-name/service-name.
      ResourceId: !Sub "service/${ClusterName}/${ServiceName}"

      # Keep at least two tasks running for availability.
      MinCapacity: 2

      # Limit maximum scale-out to control cost and protect dependencies.
      MaxCapacity: 10

      RoleARN: !Sub "arn:aws:iam::${AWS::AccountId}:role/aws-service-role/ecs.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_ECSService"

  ApiRequestScalingPolicy:
    Type: AWS::ApplicationAutoScaling::ScalingPolicy
    Properties:
      PolicyName: !Sub "${ServiceName}-request-count-scaling"
      PolicyType: TargetTrackingScaling
      ScalingTargetId: !Ref ApiScalableTarget

      TargetTrackingScalingPolicyConfiguration:
        # Request-count scaling is often better than CPU for HTTP APIs.
        PredefinedMetricSpecification:
          PredefinedMetricType: ALBRequestCountPerTarget
          ResourceLabel: "app/example-alb/1234567890abcdef/targetgroup/example-targets/abcdef1234567890"

        # Tune this value from load testing and latency targets.
        TargetValue: 800

        # Scale out quickly when traffic rises.
        ScaleOutCooldown: 60

        # Scale in more slowly to avoid capacity oscillation.
        ScaleInCooldown: 300

Important: the ResourceLabel value for ALB request count scaling must match the actual ALB and target group resource label. In production, generate or pass it from infrastructure outputs instead of hardcoding it.

Deployment Script

The deployment script below builds an immutable image, pushes it to ECR, updates the CloudFormation stack, and waits for the ECS service to stabilize.

#!/usr/bin/env bash
set -euo pipefail

AWS_REGION="us-east-1"
AWS_ACCOUNT_ID="123456789012"
ECR_REPOSITORY="orders-api"
STACK_NAME="orders-api-ecs"
CLUSTER_NAME="prod-ecs"
SERVICE_NAME="orders-api"

# Use Git SHA as an immutable image tag for rollback and auditability.
IMAGE_TAG="$(git rev-parse --short HEAD)"
IMAGE_URI="${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${ECR_REPOSITORY}:${IMAGE_TAG}"

echo "Building Docker image: ${IMAGE_URI}"

# Authenticate Docker to Amazon ECR for the target AWS account and region.
aws ecr get-login-password --region "${AWS_REGION}" \
  | docker login \
      --username AWS \
      --password-stdin "${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com"

# Build the image from the current source tree.
docker build -t "${IMAGE_URI}" .

# Push the immutable image so ECS can pull it during deployment.
docker push "${IMAGE_URI}"

echo "Deploying CloudFormation stack: ${STACK_NAME}"

# Updating ContainerImage creates a new task definition revision and updates the ECS service.
aws cloudformation deploy \
  --region "${AWS_REGION}" \
  --stack-name "${STACK_NAME}" \
  --template-file ecs-deployment-autoscaling.yaml \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides \
      ClusterName="${CLUSTER_NAME}" \
      ServiceName="${SERVICE_NAME}" \
      ContainerImage="${IMAGE_URI}"

echo "Waiting for ECS service to stabilize..."

# Wait until ECS finishes replacing old tasks and the service reaches steady state.
aws ecs wait services-stable \
  --region "${AWS_REGION}" \
  --cluster "${CLUSTER_NAME}" \
  --services "${SERVICE_NAME}"

echo "Deployment completed successfully: ${IMAGE_URI}"

Production Note: deployment scripts should fail fast. If the ECS service does not stabilize, CI/CD should stop instead of continuing with a partially healthy release.

Common Mistakes

Most ECS deployment and scaling mistakes come from treating releases and capacity as separate concerns. In production, they interact constantly.

  • Deploying with one task and expecting zero-downtime releases.
  • Using weak health checks that mark broken tasks as healthy.
  • Using heavy health checks that fail because of slow dependencies.
  • Setting maximum percent too low and forcing ECS to stop old tasks too early.
  • Scaling APIs only by CPU when request count is the better signal.
  • Scaling workers only by CPU while queue age keeps increasing.
  • Allowing fast scale-in that causes oscillation during traffic changes.
  • Ignoring deployment rollback until the first failed production release.
  • Forgetting that EC2 launch type needs spare cluster capacity during deployments.
  • Using autoscaling to hide application performance problems instead of fixing bottlenecks.

Production Checklist

A production ECS service should deploy safely, scale predictably, and recover quickly from bad releases.

  • Use immutable image tags for every deployment.
  • Run at least two tasks for production APIs.
  • Use rolling deployments as the default strategy for standard services.
  • Use blue/green or canary deployments for high-risk services.
  • Set minimum healthy percent to preserve availability during releases.
  • Set maximum percent high enough to allow temporary deployment capacity.
  • Enable deployment circuit breaker and rollback.
  • Use lightweight health checks that reflect real application readiness.
  • Scale APIs using request count, CPU, memory, or custom latency-aware metrics.
  • Scale workers using queue depth or oldest message age.
  • Use slower scale-in cooldowns than scale-out cooldowns.
  • Set maximum capacity to control cost and protect downstream dependencies.
  • Test rollback behavior before production incidents.
  • Monitor deployment events, task restarts, latency, 5xx rate, and scaling activity.

Conclusion

Amazon ECS deployment strategies and auto scaling control how services evolve under change and load. Deployments replace old task revisions with new ones, while auto scaling adjusts task count based on traffic, resource usage, backlog, or schedules.

Rolling deployments are a strong default for most ECS services. Blue/green and canary strategies are better for high-risk systems that need tighter release control. Auto scaling should follow the real bottleneck: request pressure for APIs, backlog age for workers, and resource utilization when CPU or memory is the limiting factor.

Key Takeaway: reliable ECS operations require both safe deployments and correct scaling signals. A service should roll out new versions without losing healthy capacity and scale based on the metric that best represents user impact or processing delay.

Continue with ECS articles that explain the runtime model, compute choices, networking, workload patterns, and production best practices.

Comments (0)