What Is Zero-Downtime Deployment?

Achieving zero downtime requires more than a rolling update or a load balancer. New instances must become ready before receiving requests, old instances must finish in-flight work, and both application versions must remain compatible with shared databases, caches, and message formats. A reliable release also needs enough spare capacity, production monitoring, and a rollback strategy that accounts for changes to persistent data.

Table of Contents

• How Zero-Downtime Deployment Works
• Deployment Strategies
• Readiness, Graceful Shutdown, and Traffic Draining
• Database Migrations Without Downtime
• Production Design Example
• Ready-to-Use Example: FastAPI and Kubernetes
• Monitoring and Rollback
• Common Mistakes
• Production Checklist
• Frequently Asked Questions
• Conclusion

How Zero-Downtime Deployment Works

Imagine an API running six instances behind a load balancer. The application receives 1,500 requests per second, and each instance can reliably handle 300 requests per second. Stopping all six instances to deploy new code would make the service unavailable until the replacements finish starting.

A zero-downtime release maintains enough healthy serving capacity throughout the transition. Depending on the deployment strategy, new instances may be created before old instances are removed, or an entirely separate environment may be prepared before traffic is switched.

The Deployment Lifecycle

A typical deployment follows this sequence:

1. Build: Create an immutable application image and run automated tests.
2. Prepare: Apply compatible schema changes and provision replacement capacity.
3. Start: Launch new-version instances without immediately sending them traffic.
4. Verify: Wait until initialization and readiness checks succeed.
5. Route: Direct eligible requests to the new instances.
6. Drain: Stop assigning new requests to old instances while allowing existing work to finish.
7. Retire: Remove the old instances after the shutdown grace period.
8. Validate: Confirm that application metrics and business outcomes remain healthy.

The important invariant is that removing old capacity must not reduce the service below its required serving capacity. For example, if five instances are necessary to handle peak traffic, a deployment that temporarily leaves only four healthy instances is not safe simply because those four are still running.

Zero Downtime Does Not Mean Zero Failures

Zero-downtime deployment means the release process should not make the application unavailable. It does not guarantee that every individual request succeeds under all circumstances.

Network interruptions, dependency failures, client disconnections, and unrelated infrastructure incidents can still occur. The operational objective is to avoid deployment-induced outages and keep the service within its availability and latency objectives.

For a broader explanation of redundancy and capacity planning, see Designing Highly Available Cloud Systems.

Deployment Strategies

Several release strategies can support zero-downtime deployment. They differ in how new instances are introduced, how production traffic is shifted, and how quickly a release can be reversed.

Rolling Updates

Rolling deployment is a common default for replicated stateless services. A controller starts replacement instances in batches, waits for them to become ready, and removes old instances.

Kubernetes supports this natively through Deployments. The maxSurge setting controls temporary extra pods, while maxUnavailable limits how much desired capacity can be unavailable during the rollout.

See What Is Rolling Deployment? for the full architecture and trade-offs.

Blue-Green Releases

Blue-green deployment maintains two environments. The new environment is prepared and validated before the routing layer switches traffic away from the old environment.

This provides stronger release isolation and potentially faster traffic rollback, but the old environment must remain operational and compatible with any shared data changes.

See What Is Blue-Green Deployment? for the full architecture and trade-offs.

Canary Releases

Canary deployment gradually exposes the new version to real traffic, often starting at 1% or 5%. The deployment controller evaluates error rates, latency, and other metrics before increasing exposure.

Canary releases are especially valuable when the main risk is an application regression that ordinary readiness checks cannot detect. They still require safe shutdown and data compatibility, just like rolling deployments.

See What Is Canary Deployment? for the full architecture and trade-offs.

Readiness, Graceful Shutdown, and Traffic Draining

Most deployment-related interruptions happen at the boundary between application lifecycle and traffic routing. An instance may be technically running but unable to serve requests, or it may be terminating while a load balancer still has active connections to it.

Correct startup and shutdown behavior is therefore as important as the deployment strategy itself.

Readiness vs. Liveness

Readiness answers whether an instance should receive traffic. Liveness answers whether the process is healthy enough to continue running.

A readiness probe should fail while required initialization is incomplete. A liveness probe should not restart a healthy process merely because an optional dependency is temporarily unavailable.

For example, a FastAPI instance may be alive while its application configuration or required connection pool is still initializing. It should not enter the load balancer's healthy target set until it can serve requests.

See Health Checks, Readiness, and Liveness Probes for detailed design guidance.

Graceful Shutdown

When Kubernetes terminates a pod, the application normally receives SIGTERM. The application server should stop accepting new work and allow existing requests to complete within a bounded grace period.

The termination process is not perfectly synchronized with every proxy and load balancer. Endpoint updates may take time to propagate, so a few new connections may still reach an instance during shutdown.

Long-running requests, WebSockets, gRPC streams, and background tasks need explicit shutdown policies. Some operations may be allowed to finish, while others should be cancelled and retried safely elsewhere.

Connection Draining

Connection draining removes an instance from new request routing while preserving existing work for a limited time.

For a short HTTP request, a few seconds may be sufficient. For a streaming response or WebSocket connection, the application may need to notify clients and ask them to reconnect rather than waiting indefinitely.

The load balancer's drain timeout, application server's graceful shutdown timeout, and platform termination grace period must be coordinated.

For related routing patterns, see Traffic Routing Strategies for Zero-Downtime Deployments.

Database Migrations Without Downtime

Application instances are often easier to replace than persistent data structures. A database migration can break an otherwise healthy release if it changes a schema that the previous application version still expects.

During a zero-downtime deployment, assume that old and new application versions may access the same database concurrently. Design schema changes around that overlap.

Expand-and-Contract Migrations

Suppose the application currently stores a customer's name in full_name, but a new version needs separate first_name and last_name fields.

Dropping full_name immediately is unsafe because old instances may still read and write it. Instead, use an expand-and-contract sequence:

1. Add the new columns while keeping the existing column.
2. Deploy code capable of maintaining both representations during the transition.
3. Backfill historical rows in manageable batches.
4. Validate consistency between the old and new representations.
5. Switch reads to the new columns after all required code is compatible.
6. Remove the old column in a later release.

PostgreSQL Schema Example

The expansion can begin with an additive migration:

ALTER TABLE users
    ADD COLUMN first_name TEXT,
    ADD COLUMN last_name TEXT;

Adding nullable columns without defaults is generally inexpensive in PostgreSQL, but the statement still needs a table lock. Long-running transactions can delay it, and even short migrations should be executed with appropriate lock timeouts.

For example, a migration runner can use:

BEGIN;

SET LOCAL lock_timeout = '2s';

ALTER TABLE users
    ADD COLUMN preferred_name TEXT;

COMMIT;

If the lock cannot be acquired within two seconds, the statement fails instead of waiting indefinitely and potentially blocking other work behind it.

Backfills should be processed in batches, with careful handling of concurrent writes. A backfill alone does not solve compatibility if old instances continue updating only the legacy column.

Compatibility Beyond the Database

Old and new versions may also share Redis cache values, queue messages, serialized sessions, and API responses.

Changing a cached object from a string to a nested JSON structure, for example, can break old instances even when the database schema remains compatible.

Prefer additive contract changes and versioned formats where necessary. See API Versioning and Backward Compatibility for related principles.

Production Design Example

Consider a SaaS API deployed to Amazon EKS across three availability zones. The service runs 12 FastAPI pods and handles approximately 2,400 requests per second, with PostgreSQL for persistent storage and Redis for caching.

The release must preserve API availability while updating application code and introducing a backward-compatible database field.

Production Architecture

The system includes:

• Application Load Balancer: Routes incoming requests to healthy Kubernetes targets.
• Kubernetes Deployment: Manages replicated FastAPI pods and rolling updates.
• Readiness probes: Keep uninitialized pods out of request routing.
• Pod disruption controls: Help preserve availability during supported voluntary disruptions.
• PostgreSQL: Stores application data using backward-compatible schema migrations.
• Redis: Holds shared cache entries that must remain readable across versions.
• Metrics and tracing: Measure latency, errors, saturation, and downstream failures.

Capacity During the Rollout

Suppose each pod can reliably handle 250 requests per second. Twelve pods provide a theoretical aggregate capacity of 3,000 requests per second under balanced load.

The Deployment uses maxSurge: 3 and maxUnavailable: 0, allowing three additional non-terminating pods during normal rollout progression.

However, the additional pods also consume resources. If every pod opens a database pool of 15 connections, the maximum normal rollout footprint of 15 pods can potentially consume 225 PostgreSQL connections before accounting for background workers and other services.

Deployment capacity planning must include database limits, startup bursts, cache warming, and the possibility that an availability zone becomes unhealthy during the release.

Production Release Sequence

The deployment pipeline uses the following sequence:

1. Apply the additive PostgreSQL migration.
2. Build and publish an immutable application image.
3. Run integration tests against the expanded schema.
4. Deploy the new application version using a rolling update.
5. Wait for new pods to become ready before removing old capacity.
6. Compare error rates, P95 latency, and dependency saturation against the previous version.
7. Complete the rollout and retain a rollback window.
8. Perform destructive schema cleanup only in a later release.

If the new version introduces a slow query, Kubernetes may still report a successful rollout. Application-level monitoring must detect that regression independently of readiness probes.

Ready-to-Use Example: FastAPI and Kubernetes

This example creates a small FastAPI service with a dedicated readiness endpoint, a Kubernetes rolling update configuration, and a deployment workflow that can be tested on a local cluster.

The application uses the FastAPI lifespan API for initialization and cleanup. Kubernetes controls replacement capacity, while Uvicorn handles graceful request shutdown within its configured timeout.

1. FastAPI Application

Create app.py:

import asyncio
import os
from contextlib import asynccontextmanager

from fastapi import FastAPI
from fastapi.responses import JSONResponse


VERSION = os.getenv("APP_VERSION", "v1")


@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.ready = False

    # Replace this with real resource initialization.
    await asyncio.sleep(0.5)

    app.state.ready = True

    try:
        yield
    finally:
        app.state.ready = False

        # Close database pools and external clients here.


app = FastAPI(lifespan=lifespan)


@app.get("/live")
async def live():
    return {"status": "alive"}


@app.get("/ready")
async def ready():
    if not app.state.ready:
        return JSONResponse(
            {"status": "not_ready"},
            status_code=503,
        )

    return {"status": "ready"}


@app.get("/")
async def index():
    return {
        "service": "zero-downtime-demo",
        "version": VERSION,
    }

The readiness endpoint returns a non-successful response until initialization completes. The liveness endpoint is deliberately simple and does not depend on external services.

For a real application, the lifespan handler should initialize and close actual resources. The readiness endpoint should reflect serving capability without performing unnecessarily expensive dependency checks on every probe.

2. Docker Image

Create requirements.txt:

fastapi
uvicorn

Then create the Dockerfile:

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

ARG APP_VERSION=v1
ENV APP_VERSION=${APP_VERSION}

EXPOSE 8000

CMD ["uvicorn", "app:app", \
     "--host", "0.0.0.0", \
     "--port", "8000", \
     "--timeout-graceful-shutdown", "25"]

Build the first version:

docker build \
  --build-arg APP_VERSION=v1 \
  -t zero-downtime-api:v1 .

For local Kubernetes testing, make the image available to your cluster. With kind, for example, use kind load docker-image zero-downtime-api:v1. In production, push images to a registry and prefer immutable image digests.

3. Kubernetes Deployment

Create deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: zero-downtime-api
spec:
  replicas: 4
  revisionHistoryLimit: 5
  progressDeadlineSeconds: 180

  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0

  selector:
    matchLabels:
      app: zero-downtime-api

  template:
    metadata:
      labels:
        app: zero-downtime-api

    spec:
      terminationGracePeriodSeconds: 40

      containers:
        - name: api
          image: zero-downtime-api:v1
          imagePullPolicy: IfNotPresent

          ports:
            - containerPort: 8000

          startupProbe:
            httpGet:
              path: /live
              port: 8000
            periodSeconds: 2
            failureThreshold: 20

          readinessProbe:
            httpGet:
              path: /ready
              port: 8000
            periodSeconds: 3
            failureThreshold: 2

          livenessProbe:
            httpGet:
              path: /live
              port: 8000
            periodSeconds: 10
            failureThreshold: 3

          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 500m
              memory: 256Mi
---
apiVersion: v1
kind: Service
metadata:
  name: zero-downtime-api
spec:
  selector:
    app: zero-downtime-api

  ports:
    - port: 80
      targetPort: 8000

  type: ClusterIP

The Deployment keeps four desired replicas available during a healthy rollout and permits one additional pod while replacing instances.

The 40-second termination grace period gives the application server time to finish active requests before Kubernetes forcibly terminates the container. This is a baseline configuration, not a guarantee that every long-running connection will finish.

4. Deploy and Test

Apply the manifest and wait for the initial rollout:

kubectl apply -f deployment.yaml

kubectl rollout status deployment/zero-downtime-api

kubectl get pods -l app=zero-downtime-api

Forward the Service port:

kubectl port-forward service/zero-downtime-api 8080:80

In another terminal, verify the application:

curl -i http://127.0.0.1:8080/
curl -i http://127.0.0.1:8080/ready

The first endpoint returns the application version, while the second confirms readiness.

5. Perform a Rolling Update

Build the new version:

docker build \
  --build-arg APP_VERSION=v2 \
  -t zero-downtime-api:v2 .

Load the image into your local cluster or publish it to the configured registry. Then update the Deployment:

kubectl set image deployment/zero-downtime-api \
  api=zero-downtime-api:v2

kubectl rollout status deployment/zero-downtime-api

Watch pods being replaced:

kubectl get pods -l app=zero-downtime-api --watch

In another terminal, send repeated requests:

while true; do
  curl -fsS http://127.0.0.1:8080/ \
    || echo "REQUEST FAILED"

  sleep 0.2
done

You should observe responses transitioning from v1 to v2 as the rollout progresses.

This is a smoke test, not a production-grade availability test. Kubernetes port forwarding can behave differently from an ingress controller or external load balancer, so validate real routing and connection draining separately.

6. Roll Back

If the new version introduces a regression, restore the previous Deployment revision:

kubectl rollout undo deployment/zero-downtime-api

kubectl rollout status deployment/zero-downtime-api

The controller performs another rollout to restore the previous pod template. It does not reverse database changes, published events, or external side effects.

After testing, remove the example resources with kubectl delete -f deployment.yaml.

Monitoring and Rollback

Deployment orchestration tells you whether the desired instances are running. It does not necessarily tell you whether the new application version is behaving correctly.

For example, a new version might pass every readiness probe while increasing P95 latency from 150 ms to 800 ms because of a slower database query. The rollout may complete successfully even though user experience has deteriorated.

Metrics to Monitor

Monitor the following signals by application version or deployment ID:

• Availability: Successful request rate and ready replica count
• Errors: HTTP 5xx responses, exceptions, and failed operations
• Latency: P50, P95, and P99 request duration
• Resource saturation: CPU, memory, connection pools, and queue depth
• Dependency health: Database latency, cache errors, and downstream timeouts
• Business outcomes: Successful checkouts, completed jobs, and other critical operations

Comparing metrics by version is important during rolling and canary releases because the stable version can hide regressions in aggregate dashboards.

For additional guidance, see Metrics That Actually Matter.

Automated Rollback

A deployment pipeline can automatically restore the previous revision if Kubernetes reports a rollout failure.

#!/usr/bin/env bash
set -euo pipefail

DEPLOYMENT="zero-downtime-api"
NAMESPACE="default"

if ! kubectl rollout status \
  deployment/"$DEPLOYMENT" \
  -n "$NAMESPACE" \
  --timeout=180s; then

  echo "Rollout failed. Restoring previous revision."

  kubectl rollout undo \
    deployment/"$DEPLOYMENT" \
    -n "$NAMESPACE"

  kubectl rollout status \
    deployment/"$DEPLOYMENT" \
    -n "$NAMESPACE" \
    --timeout=180s

  exit 1
fi

echo "Kubernetes rollout completed."

This script detects rollout failures, not business regressions. Production release automation should also evaluate application metrics and synthetic tests before marking the release successful.

GitOps environments require additional coordination: a rollback performed directly with kubectl may be overwritten if the desired configuration in Git still references the failed image.

Common Mistakes

Most zero-downtime deployment failures come from gaps between application behavior, infrastructure routing, and shared-state compatibility.

1. Confusing Liveness with Readiness

A running process may still be initializing required resources. Sending traffic to it too early can cause failures even when its liveness probe succeeds.

Use separate readiness and liveness checks with different responsibilities.

2. Terminating Instances Too Quickly

Removing an instance before its active requests complete can produce connection resets and failed responses.

Coordinate graceful shutdown, connection draining, and platform termination timeouts.

3. Applying Breaking Database Migrations

Old and new versions frequently run at the same time. Dropping a column or changing a shared message format can break instances that have not yet been replaced.

Use expand-and-contract migrations and delay destructive changes until all dependent versions are retired.

4. Ignoring Deployment-Time Capacity

Extra pods consume CPU, memory, database connections, and network capacity. Starting many replacements simultaneously can overload shared dependencies.

Plan for startup bursts and temporary capacity increases, not just steady-state traffic.

5. Assuming Rollback Reverses Everything

Restoring a previous container image does not undo database writes, sent emails, published messages, or external API calls.

Design rollback together with data recovery and idempotent processing where appropriate.

Production Checklist

Use this checklist before enabling zero-downtime releases for a production service.

• Capacity: Maintain enough healthy instances during every rollout stage.
• Readiness: Keep new instances out of traffic routing until initialization completes.
• Shutdown: Allow active requests and owned background work to finish safely.
• Traffic routing: Verify endpoint removal, connection draining, and load balancer behavior.
• Compatibility: Support mixed application versions and backward-compatible shared data.
• Database migrations: Avoid blocking schema changes and use staged backfills.
• Resource limits: Account for surge pods and dependency connection limits.
• Monitoring: Track version-specific errors, latency, saturation, and business outcomes.
• Rollback: Test previous-version restoration and define recovery for persistent changes.
• Automation: Stop or reverse releases when infrastructure or application health deteriorates.

Frequently Asked Questions

Zero-downtime deployment is a common production requirement, but its guarantees depend on application architecture, infrastructure behavior, and compatibility between versions.

Is True Zero-Downtime Deployment Possible?

Applications can be designed to remain available throughout a release, but no deployment strategy guarantees that every request will succeed under every possible failure condition.

The practical objective is to prevent deployment-induced outages and maintain availability and latency targets.

Does Kubernetes Guarantee Zero Downtime?

No. Kubernetes provides rolling updates, readiness probes, and lifecycle controls, but applications must use them correctly.

Insufficient capacity, incorrect readiness checks, abrupt shutdown, or incompatible database changes can still interrupt users.

Can Database Migrations Be Performed Without Downtime?

Many database changes can be performed without application downtime using additive migrations, staged backfills, and backward-compatible application code.

Some operations still require locks or specialized migration techniques, particularly on large tables.

Is Rolling or Blue-Green Deployment Better?

Rolling deployment generally uses less temporary infrastructure and is straightforward for replicated stateless services. Blue-green deployment provides stronger environment separation and can offer faster traffic rollback while the previous environment remains available.

The choice depends on cost, deployment frequency, rollback requirements, and application architecture.

Do Serverless Applications Need Zero-Downtime Deployment Strategies?

Yes. Serverless platforms manage much of the infrastructure lifecycle, but application compatibility and release safety remain important.

For example, AWS Lambda versions and aliases can support controlled traffic shifting. Database migrations, event formats, and external side effects still need careful planning.

Conclusion

Zero-downtime deployment keeps applications available while new versions are introduced into production. It relies on maintaining sufficient healthy capacity, coordinating traffic routing with application startup and shutdown, and ensuring that old and new versions can safely coexist.

Rolling, blue-green, and canary strategies provide different ways to manage the transition, but none eliminates the need for backward-compatible database migrations, reliable monitoring, and tested rollback procedures.

Key Takeaway: Zero-downtime deployment is an end-to-end reliability practice, not a single Kubernetes setting or load-balancer feature. Start new instances before removing required capacity, drain old instances safely, preserve compatibility between versions, and validate releases against real production health metrics.

More Articles to Read

Explore these related articles for deeper coverage of deployment strategies and production reliability:

• Reliability Best Practices for Production Systems