Traffic Routing Strategies for Zero-Downtime Deployments

Deploying a new application version without downtime requires more than starting new instances. Production traffic must move from the old version to the new version without routing requests to unready instances, interrupting in-flight work, or exposing an unhealthy release to every client.

Load balancers provide the traffic-control layer that makes this possible. Rolling deployments, blue-green deployments, canary releases, and weighted traffic shifting use different routing strategies, but they share the same core requirement: traffic must move between versions in a controlled and reversible way.

Table of Contents

• Why Traffic Routing Matters During Deployments
• Rolling Deployments
  • Advantages
  • Disadvantages
  • When to Use
  • Example
• Blue-Green Deployments
  • Advantages
  • Disadvantages
  • When to Use
  • Example
• Canary and Weighted Traffic Routing
  • Advantages
  • Disadvantages
  • When to Use
  • Example
• Readiness, Draining, and Safe Traffic Transition
• Production Design Example
• Common Mistakes
• Production Checklist
• Conclusion
• Key Takeaway
• More Articles to Read

Why Traffic Routing Matters During Deployments

A deployment changes the set of application instances that should receive traffic. During the transition, old and new versions may run simultaneously.

Before deployment

Load Balancer
     |
     +----> v1
     +----> v1
     +----> v1


During deployment

Load Balancer
     |
     +----> v1
     +----> v1
     +----> v2


After deployment

Load Balancer
     |
     +----> v2
     +----> v2
     +----> v2

The dangerous periods are the transitions between these states.

A new instance may have started its process but still be loading configuration, warming caches, establishing database connections, running initialization logic, or waiting for dependencies. Routing production traffic too early creates deployment errors even when the new version itself is correct.

Removing old instances creates the opposite problem. An instance may still have active HTTP requests, WebSocket connections, uploads, or other work when the deployment system terminates it.

A safe deployment therefore needs two controlled operations:

1. Admission: determine when a new instance becomes eligible for production traffic.
2. Removal: stop sending new traffic before terminating an old instance.

These operations make application lifecycle and traffic lifecycle separate concepts. Process running does not mean ready for traffic, and removal from traffic does not mean safe to terminate immediately.

For the broader relationship between load balancers, health checks, and backend pools, see Load Balancing Explained: Distributing Traffic at Scale.

Rolling Deployments

A rolling deployment gradually replaces old application instances with new ones while both versions temporarily serve production traffic.

The load balancer continuously adjusts its backend pool as new instances become ready and old instances are drained.

Step 1
[v1] [v1] [v1] [v1]

Step 2
[v2] [v1] [v1] [v1]

Step 3
[v2] [v2] [v1] [v1]

Step 4
[v2] [v2] [v2] [v1]

Step 5
[v2] [v2] [v2] [v2]

Rolling deployments are particularly common with container orchestration because they require relatively little additional infrastructure.

Advantages

• Resource efficient: only limited additional capacity is required during deployment.
• Gradual replacement: the entire fleet is not replaced simultaneously.
• Simple infrastructure: no permanent second environment is required.
• Good orchestration support: platforms such as Kubernetes can automate instance replacement and readiness.

Disadvantages

The largest architectural constraint is that multiple application versions coexist.

During deployment:

Client A --> v1
Client B --> v2
Client C --> v1
Client D --> v2

Database schemas, APIs, message formats, cache structures, and shared state must remain compatible across both versions.

A deployment that changes a database column and immediately assumes every application instance uses the new schema can break old instances still serving traffic.

Rollback can also be slower than a simple traffic switch because the orchestration system may need to replace instances again.

When to Use

Rolling deployments work well when:

• application versions can coexist safely;
• database changes are backward compatible;
• additional full-environment capacity is undesirable;
• instances are relatively fast to create and replace;
• the deployment platform provides reliable readiness and draining behavior.

Example

A Kubernetes Deployment can limit how quickly old instances disappear and new instances are introduced:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 6

  strategy:
    type: RollingUpdate
    rollingUpdate:
      # Keep normal capacity during deployment.
      maxUnavailable: 0

      # Allow limited temporary extra capacity.
      maxSurge: 2

  template:
    spec:
      containers:
        - name: api
          image: example/api:v2

          readinessProbe:
            httpGet:
              path: /ready
              port: 8000
            periodSeconds: 5
            failureThreshold: 2

          lifecycle:
            preStop:
              exec:
                # Give routing infrastructure time to stop
                # forwarding new requests before termination.
                command: ["/bin/sh", "-c", "sleep 10"]

      # Allow existing work to finish before forceful termination.
      terminationGracePeriodSeconds: 60

The exact configuration depends on request duration, endpoint propagation time, startup behavior, and the orchestration environment. Arbitrary sleep periods should not replace proper draining mechanisms when the platform provides them.

Blue-Green Deployments

A blue-green deployment maintains two separate application environments. One serves production traffic while the other receives the new release.

The new Green environment can start, initialize, and undergo health verification before receiving normal production traffic.

                    Load Balancer
                         |
                         |
                    100% traffic
                         |
                         v
                  +-------------+
                  | Blue - v1   |
                  +-------------+

                  +-------------+
                  | Green - v2  |
                  +-------------+
                    no traffic

When the release is ready, routing changes:

Before switch:

Blue v1  = 100%
Green v2 =   0%


After switch:

Blue v1  =   0%
Green v2 = 100%

The old environment can remain available temporarily for rollback.

Advantages

• Fast traffic switch: production can move between complete environments.
• Simple application rollback: traffic can often return to the previous environment quickly.
• Pre-production validation: the new environment can become fully ready before receiving normal traffic.
• Version isolation: application instances from different releases do not need to share one target pool.

Disadvantages

The main cost is infrastructure duplication. During deployment, both environments may need enough capacity to serve production.

Rollback is also not automatically safe just because traffic routing can be reversed. If the new release writes data using an incompatible schema or changes external state, the old version may no longer understand that state.

Traffic rollback and data rollback are separate problems.

Long-lived connections also complicate the switch. Existing connections to Blue may remain there while new connections go to Green, meaning the transition is not always instantaneous from the application's perspective.

When to Use

Blue-green deployments are useful when:

• fast application rollback is important;
• enough infrastructure capacity exists for two environments;
• releases benefit from full-environment validation;
• traffic can be switched through a load balancer or routing layer;
• shared data remains compatible with both versions during the rollback window.

Example

At the load-balancing layer, Blue and Green can exist as separate target groups:

                  Public Endpoint
                        |
                        v
                  Load Balancer
                    /       \
                   /         \
                  v           v
           Target Group    Target Group
               Blue           Green
                v1              v2
             /     \          /     \
          App 1   App 2    App 3   App 4

The deployment system validates Green before modifying the production routing rule. After the switch, Blue remains registered but receives no new production traffic until the rollback window expires.

Canary and Weighted Traffic Routing

A canary deployment sends a small portion of production traffic to the new version before expanding the rollout.

The purpose is not merely to deploy gradually. The purpose is to limit the blast radius while observing the new version under real production traffic.

Initial:
v1 = 100%
v2 =   0%

Canary:
v1 =  95%
v2 =   5%

Expansion:
v1 =  75%
v2 =  25%

Later:
v1 =  50%
v2 =  50%

Complete:
v1 =   0%
v2 = 100%

A canary can reveal failures that staging environments often miss, including production data patterns, unusual request combinations, real dependency behavior, cache effects, and scale-related problems.

Advantages

• Limited blast radius: only part of production traffic initially reaches the new release.
• Real traffic validation: the new version is evaluated against actual workloads.
• Progressive rollout: exposure can increase as confidence grows.
• Metric-driven decisions: rollout can stop when latency or error rates degrade.

Disadvantages

Canary deployments require substantially better observability than simple traffic switching. Metrics must distinguish the old version from the new one.

This aggregate metric is insufficient:

Overall error rate: 0.7%

The useful comparison is:

v1 error rate: 0.2%
v2 error rate: 8.4%

v1 p95 latency: 120 ms
v2 p95 latency: 680 ms

If only 5% of traffic reaches v2, severe canary failures can remain hidden inside apparently healthy global metrics.

Low traffic volume creates another challenge. A 1% canary may not receive enough requests to produce statistically useful results for a low-traffic service.

When to Use

Canary routing works well when:

• releases carry meaningful production risk;
• traffic volume is high enough for useful measurements;
• metrics can be segmented by version;
• versions can safely coexist;
• routing weights can be changed reliably;
• automated or operator-driven rollback criteria are defined.

Example

A routing system can represent a canary using weighted target groups:

routing:
  target_groups:
    - name: production-v1
      weight: 95

    - name: canary-v2
      weight: 5

  canary_validation:
    # Rollout should evaluate the new version independently.
    metrics:
      - error_rate
      - p95_latency
      - p99_latency

    rollback:
      error_rate_percent: 2
      p95_latency_ms: 500

The exact thresholds should come from service objectives and historical behavior rather than arbitrary universal values.

Weighted routing is not necessarily deterministic for small request samples. A 5% weight describes the expected distribution over enough traffic, not a guarantee that exactly five of every hundred requests reach the canary.

Readiness, Draining, and Safe Traffic Transition

Rolling, blue-green, and canary strategies differ in how versions receive traffic, but they all depend on a safe backend lifecycle.

The ideal lifecycle separates startup, readiness, serving, draining, and termination:

Start
  |
  v
Initialize
  |
  v
Not Ready
  |
  | readiness succeeds
  v
Ready
  |
  v
Receive Traffic
  |
  | deployment / scale-in
  v
Draining
  |
  | in-flight work completes
  v
Terminate

Readiness controls traffic admission. The application should not enter the load-balancer pool until it can process production requests.

A readiness endpoint might verify essential initialization without turning every downstream dependency into a hard requirement:

from fastapi import FastAPI, Response, status

app = FastAPI()

application_initialized = False


@app.get("/ready")
def ready(response: Response):
    if not application_initialized:
        response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
        return {"ready": False}

    return {"ready": True}

When removing an instance, the sequence should run in reverse from a traffic perspective:

1. Mark the instance unavailable for new traffic.
2. Propagate deregistration through the routing layer.
3. Allow active requests and connections to finish.
4. Terminate after the configured grace period.

Connection draining is especially important for long requests, file transfers, streaming APIs, WebSockets, and other persistent connections.

Readiness and liveness solve different problems. More about their failure semantics can be found in Health Checks, Readiness, and Liveness Probes.

Applications should also avoid unnecessary local session dependencies. If requests must return to specific instances, deployment routing becomes more difficult because removing an old instance can disrupt clients assigned to it. For a deeper explanation, see Sticky Sessions and Stateless Applications.

Production Design Example

Consider an API running across three availability zones. A new release changes important request-processing logic, so production traffic should move gradually rather than replacing the entire fleet immediately.

The architecture maintains separate target groups for the stable and candidate versions:

                           Clients
                              |
                              v
                     +----------------+
                     | Load Balancer  |
                     +----------------+
                       /            \
                      /              \
                 weighted          weighted
                  traffic           traffic
                    /                  \
                   v                    v
          +----------------+    +----------------+
          | Stable v1      |    | Canary v2      |
          | Target Group   |    | Target Group   |
          +----------------+    +----------------+
           /      |       \       /      |      \
          v       v        v     v       v       v
        Zone A  Zone B   Zone C Zone A  Zone B  Zone C

The deployment begins with v2 receiving no public traffic. New instances start, initialize, and pass readiness checks.

Traffic then progresses through predefined stages:

Each stage has a minimum observation period and evaluates v2 independently using request success rate, latency percentiles, resource saturation, dependency errors, and business-level signals relevant to the service.

If v2 exceeds rollback thresholds at 25%:

v1: 75%
v2: 25%

       |
       | v2 error threshold exceeded
       v

v1: 100%
v2:   0%

       |
       v

Investigate v2 while stable fleet serves traffic

Removing traffic from v2 is only the first rollback action. In-flight requests should drain rather than being terminated immediately.

If the release changes the database, the schema migration must support both versions throughout the coexistence and rollback window. A common approach is an expand-and-contract migration:

1. Add the new schema without removing fields required by v1.
2. Deploy application code capable of working during the transition.
3. Move traffic to v2.
4. Verify the release and complete the rollback window.
5. Remove obsolete schema only after v1 can no longer receive traffic.

This principle applies beyond databases. Message formats, API contracts, cached representations, and shared files may also need compatibility while multiple versions coexist.

Finally, the old fleet should not be destroyed the instant v2 reaches 100%. Keeping it temporarily available can provide faster application rollback, provided shared data remains compatible.

Common Mistakes

Production Checklist

• Choose the deployment strategy intentionally: match rolling, blue-green, or canary routing to release risk and infrastructure constraints.
• Separate startup from readiness: route traffic only after the application can serve production requests.
• Enable graceful draining: remove instances from new traffic before termination.
• Set realistic termination periods: account for long requests, streams, uploads, and persistent connections.
• Maintain version compatibility: allow old and new releases to coexist whenever the strategy requires it.
• Use backward-compatible data migrations: do not make rollback impossible before the new release is proven stable.
• Preserve deployment capacity: account for surge instances and draining targets when sizing the fleet.
• Segment metrics by version: compare errors, latency, resource use, and dependency behavior independently.
• Define rollback thresholds: decide acceptable failure and latency levels before production exposure.
• Use meaningful canary stages: provide enough traffic and observation time to produce useful evidence.
• Test rollback regularly: verify that routing can actually return to the previous release.
• Account for session affinity: understand whether sticky clients delay or distort traffic transitions.
• Monitor target health during rollout: detect new instances repeatedly entering and leaving the healthy pool.
• Automate repeatable transitions: avoid manual routing changes that can leave target weights or backend pools inconsistent.
• Keep the previous version available when appropriate: preserve a practical rollback path until the release is sufficiently validated.

Conclusion

Zero-downtime deployment is fundamentally a traffic-management problem. New application versions must become ready before receiving requests, old versions must stop receiving new traffic before termination, and mixed-version periods must remain compatible with shared data and dependencies.

Rolling deployments optimize infrastructure efficiency, blue-green deployments provide strong environment isolation and fast traffic switching, while canary routing limits blast radius through progressive production exposure. The correct strategy depends on release risk, capacity, compatibility requirements, and observability.

Key Takeaway

A safe deployment does not replace instances and hope traffic follows correctly; it explicitly controls when each version becomes eligible for traffic and when it can safely leave. Combine readiness, weighted routing, version-specific observability, backward compatibility, connection draining, and tested rollback paths to make deployments routine production events instead of availability risks.

More Articles to Read

• Load Balancing Explained: Distributing Traffic at Scale
• Round Robin vs Least Connections vs Consistent Hashing
• Designing Highly Available Load Balancing Architectures
• Sticky Sessions and Stateless Applications
• Global Load Balancing and Multi-Region Traffic Routing
• Load Balancing Best Practices for Production Systems