Scalability for Dummies - Part 1: Clones

Cloning is the first and often most practical way to scale a system: run more identical copies of the same application so more requests can be handled in parallel.

Table of Contents

• What Are Clones?
  • Horizontal Scaling
  • Real-World Analogy
• Why Use Clones?
  • More Capacity
  • Better Fault Tolerance
  • Elastic Scaling
• The Hard Parts
  • Statefulness
  • Sticky Sessions
  • Shared Database Bottleneck
  • Configuration Drift
• Patterns for Scaling With Clones
  • Stateless Architecture
  • Load Balancing
  • Health Checks and Graceful Shutdown
  • Autoscaling
  • Shared Caches and Rate Limits
• Architecture Example
• Docker Compose Example
• Pre-Flight Checklist
• Common Mistakes
• Conclusion

What Are Clones?

A clone is another identical instance of your application. It can be another process, container, virtual machine, or server running the same code and configuration.

Instead of making one server bigger and bigger, you add more copies of the same application and put them behind a load balancer.

Horizontal Scaling

Cloning is a form of horizontal scaling.

Vertical scaling:
1 bigger server

Horizontal scaling:
many normal servers

For example, if one API server can handle 500 requests per second, then four similar API servers may handle roughly 2,000 requests per second, assuming the database, cache, network, and downstream systems can also handle the load.

Real-World Analogy

Imagine a restaurant kitchen. If one chef cannot prepare enough meals during dinner rush, you can either try to make that chef work faster or add more chefs.

Adding more chefs is similar to adding clones. Each chef follows the same recipe, but more orders can be processed at the same time.

However, adding chefs does not solve every problem. If there is only one oven, the oven becomes the bottleneck. In software, the shared database often becomes that oven.

Why Use Clones?

More Capacity

The main reason to use clones is capacity. More application instances can handle more concurrent users, requests, jobs, or background tasks.

For example, a web API that becomes slow at 80% CPU can be scaled by adding more API containers behind the load balancer.

Better Fault Tolerance

Clones also improve availability. If one clone crashes, the load balancer can stop sending traffic to it and continue routing requests to healthy clones.

Before failure:
Load Balancer -> API 1, API 2, API 3

After API 2 fails:
Load Balancer -> API 1, API 3

The system is not perfect, but it continues working.

Elastic Scaling

Clones are useful because they can be added or removed based on demand.

For example, an e-commerce system may run 5 API instances during normal traffic and 50 instances during a holiday sale. After the sale ends, the system can scale back down.

• Simplicity: same code, same behavior, fewer surprises.
• Predictability: if one clone handles X traffic, N clones may handle roughly N times X.
• Fault isolation: one bad clone does not bring down the whole system.
• Elasticity: instances can be added or removed based on load.
• Parallel processing: multiple clones can handle work at the same time.

The Hard Parts

Cloning sounds simple, but it only works well when the application is designed for it.

Statefulness

The biggest problem is local state.

If each clone stores user sessions, uploaded files, or temporary data on its own local disk or memory, requests may break when the user is routed to a different clone.

Request 1 -> API 1 stores session locally
Request 2 -> API 2 cannot find session
User is logged out or request fails

This is why scalable applications usually move state outside the application instance.

Sticky Sessions

Sticky sessions force the load balancer to send the same user to the same clone.

This can hide state problems, but it creates new problems. Some clones may become hotter than others, users are affected when their assigned clone fails, and deployments become harder.

Sticky sessions are sometimes useful as a temporary solution, but they should not be the foundation of a scalable architecture.

Shared Database Bottleneck

Adding application clones does not automatically scale the database.

10 API clones
  │
  ▼
1 database

If every clone sends more queries to the same primary database, the database may become the bottleneck.

For example, scaling from 5 API instances to 50 API instances can overload the database connection limit, increase lock contention, or create slow queries under load.

Configuration Drift

Clones should be identical. If one clone runs different code, different environment variables, or different dependencies, debugging becomes painful.

A common example is one container running an old build while others run a new build. Some users see one behavior, while other users see another.

Always build clones from a single versioned source of truth.

Patterns for Scaling With Clones

Stateless Architecture

A stateless application does not keep important user or business state inside the local process.

Instead, state is stored in external systems:

• Redis or Memcached for sessions and short-lived cache.
• Object storage for files and images.
• Databases for durable business data.
• Message queues for background work.

This allows any clone to handle any request.

Good:
Request can go to API 1, API 2, or API 3

Bad:
Request must always go to API 1 because session lives there

Load Balancing

A load balancer distributes traffic across clones.

It can also remove unhealthy clones from rotation, support TLS termination, and route requests based on host, path, or other rules.

Common load balancing tools include AWS ALB, AWS ELB, NGINX, HAProxy, Envoy, and Kubernetes Services.

Health Checks and Graceful Shutdown

Each clone should expose a health endpoint so the load balancer knows whether it should receive traffic.

GET /health

A clone should also shut down gracefully. During shutdown, it should stop accepting new requests, finish existing requests when possible, and then exit.

Without graceful shutdown, deployments can drop active user requests.

Autoscaling

Autoscaling automatically changes the number of clones based on metrics.

Common scaling signals include:

• CPU usage.
• Memory usage.
• Request latency.
• Queue depth.
• Number of active connections.
• Custom business metrics.

CPU is easy to measure, but it is not always the best signal. A background worker system may scale better based on queue depth. An API may scale better based on p95 latency.

Shared Caches and Rate Limits

When you add more clones, you also increase pressure on downstream systems.

A shared cache can reduce repeated expensive work. Shared rate limits can prevent clones from collectively overwhelming a database, payment provider, or third-party API.

For example, if each clone allows 100 requests per second to a third-party API, then 20 clones may send 2,000 requests per second. That may exceed the provider limit.

Rate limits should usually be global or shared, not only local per clone.

Architecture Example

A simple web API scaling from one instance to many clones can look like this:

Client
   │
   ▼
Load Balancer
   │
   ├── API Clone 1
   ├── API Clone 2
   └── API Clone N
          │
          ▼
      Redis / Cache
          │
          ▼
   Primary Database
          │
          ▼
     Read Replicas

In this design, API clones are stateless. Sessions are stored in Redis, files go to object storage, business data is stored in the database, and read replicas can help distribute read-heavy traffic.

This architecture is simple and common because it separates compute scaling from state management.

Docker Compose Example

A local Docker Compose example may look like this:

version: '3.8'

services:
  web:
    image: myapp:latest
    deploy:
      replicas: 3
    ports:
      - "8080:80"
    environment:
      - REDIS_URL=redis:6379

  redis:
    image: redis:6-alpine

In Kubernetes, the same idea appears through the replicas field in a Deployment.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
        - name: myapp
          image: myapp:latest

The important idea is the same: run multiple identical application instances.

Pre-Flight Checklist

Before scaling with clones, ask these questions:

• Is the application stateless?
• Are sessions stored outside the application instance?
• Are uploaded files stored in object storage instead of local disk?
• Are health checks implemented?
• Does the application shut down gracefully?
• Can the database handle more concurrent connections?
• Can downstream APIs handle more traffic?
• Are logs, metrics, and traces available per clone?
• Are all clones built from the same image and configuration source?

Common Mistakes

Scaling Without Observability

Adding clones without metrics is scaling blind. You may add instances and still not know whether latency improved, database load increased, or errors moved somewhere else.

Ignoring the Database

The application layer may scale horizontally, but the database may not. More clones usually mean more queries and more connections.

Use connection pooling, query optimization, caching, read replicas, and database monitoring before assuming more clones will solve everything.

Depending on Local Files

If users upload files to local disk, those files exist only on one clone. When another clone handles the next request, the file may be missing.

Use object storage such as S3 for files that must survive across clones.

Using Sticky Sessions as a Permanent Fix

Sticky sessions can work temporarily, but they reduce flexibility and make failure handling worse.

If possible, move session state into a shared store and allow any clone to serve any request.

Conclusion

Clones are one of the simplest and most reliable ways to scale an application horizontally. They work especially well for stateless services where any instance can handle any request.

However, cloning does not solve every bottleneck. The database, cache, message queue, third-party APIs, and shared resources may become overloaded as more clones generate more traffic.

The practical rule is simple:

Clone the application layer, but centralize state, monitor everything, and make sure downstream systems can handle the extra load.

In Scalability for Dummies - Part 2: Database, we will look at why databases become bottlenecks and how techniques like replication, sharding, indexing, and caching help keep systems fast and resilient.