Round Robin vs Least Connections vs Consistent Hashing

A load balancer needs more than a list of healthy servers. For every incoming request or connection, it must decide which backend should receive the traffic. That decision is controlled by the load-balancing algorithm.

Round Robin, Least Connections, and Consistent Hashing solve different traffic-distribution problems. Round Robin assumes backends and workloads are relatively uniform, Least Connections reacts to current connection distribution, and Consistent Hashing provides stable routing based on a request key.

Table of Contents

• Why Routing Algorithms Matter
• Round Robin
  • Advantages
  • Disadvantages
  • When to Use
  • Example
• Least Connections
  • Advantages
  • Disadvantages
  • When to Use
  • Example
• Consistent Hashing
  • Advantages
  • Disadvantages
  • When to Use
  • Example
• Choosing the Right Algorithm
• Production Design Example
• Common Mistakes
• Production Checklist
• Conclusion
• Key Takeaway
• More Articles to Read

Why Routing Algorithms Matter

Load balancing does not automatically mean traffic is distributed efficiently. Two backend servers can receive the same number of requests while experiencing completely different resource utilization.

Consider three application instances:

Backend A: 100 requests, average duration 20 ms
Backend B: 100 requests, average duration 500 ms
Backend C: 100 requests, average duration 40 ms

The request count is perfectly balanced, but the workload is not. Backend B spends substantially more time processing its requests.

The routing algorithm determines which signal matters when choosing a target:

• Round Robin: whose turn is next?
• Least Connections: which backend currently has fewer active connections?
• Consistent Hashing: which backend is associated with this request key?

No algorithm can perfectly infer CPU, memory pressure, database contention, downstream latency, and request complexity from a single metric. The objective is therefore not to find a universally best algorithm, but to select one whose assumptions match the workload.

For an overview of where routing algorithms fit into the complete traffic path, see Load Balancing Explained: Distributing Traffic at Scale.

Round Robin

Round Robin distributes requests sequentially across available backend servers. After reaching the final server, selection starts again from the beginning.

Backends: A, B, C

Request 1 --> A
Request 2 --> B
Request 3 --> C
Request 4 --> A
Request 5 --> B
Request 6 --> C
Request 7 --> A

The load balancer typically maintains a pointer to the next eligible backend. Selection is inexpensive because no runtime load calculation is required for every request.

If a backend becomes unhealthy, it is removed from the eligible pool:

A = healthy
B = unhealthy
C = healthy

Request 1 --> A
Request 2 --> C
Request 3 --> A
Request 4 --> C

Advantages

• Simple: backend selection requires little state or computation.
• Predictable: traffic is distributed evenly by request or connection count.
• Efficient: routing does not require continuous backend-load measurements.
• Easy to operate: behavior is straightforward during debugging and capacity analysis.

Disadvantages

Round Robin assumes that requests and backend capacity are reasonably similar. That assumption can fail when request costs vary significantly.

Request 1 --> A --> 20 ms
Request 2 --> B --> 8 seconds
Request 3 --> C --> 40 ms
Request 4 --> A --> 30 ms
Request 5 --> B --> 6 seconds

Backend B continues receiving its normal share even though previous requests are still consuming resources.

Another problem appears when backend instances have different capacities. A 2-vCPU instance and a 16-vCPU instance receiving identical traffic volumes are unlikely to operate at similar utilization.

Weighted Round Robin can address unequal backend capacity by assigning larger weights to stronger instances, but the configured weights remain static unless another system adjusts them.

When to Use

Round Robin works particularly well when:

• backend instances have similar capacity;
• request processing times are relatively uniform;
• requests are short-lived;
• applications are stateless;
• simple and predictable routing is preferred.

Typical examples include homogeneous stateless HTTP API fleets, web applications, and services where individual request costs do not differ dramatically.

Example

A simple NGINX upstream configuration uses Round Robin by default:

upstream api_backend {
    server api-1:8000;
    server api-2:8000;
    server api-3:8000;
}

server {
    listen 80;

    location / {
        proxy_pass http://api_backend;
    }
}

With three equivalent healthy servers, requests are distributed across the upstream pool without requiring application-level routing logic.

Least Connections

Least Connections routes new traffic to the backend currently handling the fewest active connections.

Suppose the current state is:

Backend A: 17 active connections
Backend B:  5 active connections
Backend C: 11 active connections

The next connection is routed to Backend B.

If several backends have the same minimum connection count, another selection mechanism such as Round Robin can break the tie.

This algorithm reacts to current connection distribution rather than assuming every previous request has already completed.

Advantages

• Runtime awareness: selection considers current connection distribution.
• Better handling of long-lived connections: busy targets naturally accumulate connections and receive less new traffic.
• Adaptive behavior: no static request-cost estimate is required.
• Useful for uneven request duration: slow requests remain represented by active connections.

Disadvantages

The major limitation is that connection count is not resource utilization.

Consider:

Backend A
10 connections
CPU usage: 95%

Backend B
100 mostly idle WebSocket connections
CPU usage: 15%

Least Connections may prefer Backend A because it has fewer connections, even though it has substantially less available CPU capacity.

Modern HTTP behavior complicates the signal further. HTTP/2 can multiplex many concurrent requests over a small number of TCP connections, while keep-alive connections can remain open even when little work is being performed.

The algorithm also requires the load-balancing layer to track connection state accurately, which creates more runtime state than basic Round Robin selection.

When to Use

Least Connections is useful when:

• connections remain active for significantly different durations;
• request duration varies substantially;
• traffic includes long-lived connections;
• backend instances are otherwise reasonably similar;
• active connection count correlates meaningfully with backend load.

Examples include application servers handling long-running requests, database proxies, TCP services, and some real-time communication workloads.

Example

NGINX can select the backend with the smallest number of active connections:

upstream api_backend {
    least_conn;

    server api-1:8000;
    server api-2:8000;
    server api-3:8000;
}

server {
    listen 80;

    location / {
        proxy_pass http://api_backend;
    }
}

The difference from Round Robin is subtle but important: backend selection now changes according to connection lifetime rather than simply rotating through the server list.

Consistent Hashing

Consistent Hashing solves a different problem. Its primary objective is not to make every backend receive exactly the same number of requests. Instead, it provides stable mapping between a routing key and a backend.

A routing key might be:

• tenant ID;
• user ID;
• cache key;
• session identifier;
• resource ID;
• request path or another stable value.

A naive hash approach could use:

backend_index = hash(key) % len(backends)

With four backends, the result maps each key to one of four positions. The problem appears when the backend count changes.

Before:
hash(key) % 4

After adding one server:
hash(key) % 5

Because the divisor changes, a large percentage of keys can suddenly map to different servers. For a distributed cache, this can invalidate much of the effective cache placement at once and cause a large increase in traffic to databases or downstream systems.

Consistent hashing reduces this disruption by placing both backends and keys into a logical hash space, commonly represented as a ring.

                   Backend A
                       |
                +------+------+
             key-1           key-8
            /                   \
     Backend D                 Backend B
            \                   /
             key-4           key-3
                +------+------+
                       |
                   Backend C

A key is hashed to a position and assigned according to the ring's lookup rule, commonly the next backend clockwise. When a backend is added or removed, only part of the key space needs to move.

Production implementations often use virtual nodes, where each physical backend owns multiple positions in the hash space. This improves distribution and reduces the chance that one backend receives a disproportionately large range.

Advantages

• Stable routing: the same key normally reaches the same backend while membership remains stable.
• Limited remapping: adding or removing a backend affects only part of the key space.
• Cache locality: repeated access to a key can reach the backend already holding relevant data.
• Useful affinity: routing can follow tenant, user, or resource ownership without maintaining a central mapping for every request.

Disadvantages

Stable mapping does not guarantee balanced load. If a small number of keys generate most traffic, the servers responsible for those keys can become hotspots.

Tenant A --> Backend 1 --> 50,000 req/s
Tenant B --> Backend 2 -->    500 req/s
Tenant C --> Backend 3 -->    700 req/s

The hash distribution may be mathematically reasonable while the traffic distribution is severely unbalanced.

Consistent hashing also introduces additional complexity around membership changes, virtual nodes, replication, failover, and overloaded keys.

When a backend disappears, its keys must move somewhere. The replacement backend may suddenly receive traffic for data that is not locally cached, creating a temporary cold-cache surge.

When to Use

Consistent Hashing is useful when stable key placement matters:

• distributed caches;
• partitioned application workloads;
• tenant-aware routing;
• stateful processing where affinity is intentional;
• systems where backend membership changes should cause minimal remapping.

It should not be selected merely as a more sophisticated replacement for Round Robin. If request affinity provides no benefit, Consistent Hashing adds complexity without solving an important problem.

Example

The basic mechanics can be illustrated with a small hash ring:

import bisect
import hashlib


class HashRing:
    def __init__(self, backends: list[str], virtual_nodes: int = 100):
        self.ring: list[tuple[int, str]] = []

        # Multiple positions per backend improve distribution.
        for backend in backends:
            for replica in range(virtual_nodes):
                value = self._hash(f"{backend}:{replica}")
                self.ring.append((value, backend))

        self.ring.sort()

    @staticmethod
    def _hash(value: str) -> int:
        digest = hashlib.sha256(value.encode()).digest()
        return int.from_bytes(digest[:8], "big")

    def get_backend(self, key: str) -> str:
        if not self.ring:
            raise RuntimeError("No backends available")

        position = self._hash(key)
        hashes = [item[0] for item in self.ring]

        # Find the first backend clockwise from the key.
        index = bisect.bisect_left(hashes, position)

        # Wrap around when the key is beyond the final ring position.
        if index == len(self.ring):
            index = 0

        return self.ring[index][1]


ring = HashRing([
    "cache-1",
    "cache-2",
    "cache-3",
])

backend = ring.get_backend("tenant:12345")
print(backend)

This example demonstrates the mapping principle. Production implementations usually optimize lookup structures and need explicit mechanisms for backend discovery, health, replication, membership changes, and hotspot management.

Choosing the Right Algorithm

The most important difference between these algorithms is what information drives the routing decision.

A useful rule of thumb is:

• Start with Round Robin when backends and requests are similar.
• Use Least Connections when active connection count meaningfully represents backend workload.
• Use Consistent Hashing when stable key-to-backend mapping is an architectural requirement.

Weighted variants can improve Round Robin and Least Connections when backend capacities differ. More advanced load balancers may also incorporate observed latency, outstanding requests, backend weights, or resource signals.

Algorithm complexity should follow workload complexity. A sophisticated routing strategy is not automatically better than a simple one.

Production Design Example

Consider a platform containing three different traffic patterns: a stateless REST API, a long-running processing service, and a distributed in-memory cache.

                         Clients
                            |
                            v
                    +---------------+
                    | Load Balancer |
                    +---------------+
                      /           \
                     /             \
          REST API traffic     Processing traffic
                  |                    |
            Round Robin        Least Connections
                  |                    |
          +-------+-------+      +-----+-----+
          |       |       |      |           |
        API 1   API 2   API 3  Worker 1   Worker 2


Application cache requests
          |
          v
   Consistent Hashing
          |
    +-----+-----+
    |     |     |
 Cache 1 Cache 2 Cache 3

The REST API uses Round Robin because application instances are homogeneous, requests are relatively short, and no request requires affinity.

The processing service uses Least Connections because requests can remain active for significantly different periods. A worker already handling several long-running connections should receive less new traffic than an idle worker.

The cache tier uses Consistent Hashing because repeated requests for the same key benefit from reaching the same cache node. When another cache node is introduced, only part of the key space should move.

This architecture demonstrates an important production principle: there is no requirement for every layer of a system to use the same load-balancing algorithm. Routing should match the behavior of each workload.

Health checking remains independent from algorithm selection. Round Robin should not route to an unhealthy target, Least Connections should not select a failed server merely because it has zero connections, and Consistent Hashing needs a strategy for remapping keys when the selected target disappears.

For architectures that need the load-balancing layer itself to survive infrastructure failures, see Designing Highly Available Load Balancing Architectures.

Common Mistakes

Production Checklist

• Profile workload shape: measure request duration, connection lifetime, and resource consumption before selecting an algorithm.
• Validate backend symmetry: determine whether targets actually have comparable CPU, memory, and throughput capacity.
• Start with the simplest sufficient algorithm: avoid routing complexity without a concrete production requirement.
• Use weights intentionally: represent meaningful capacity differences rather than arbitrary tuning.
• Verify connection semantics: understand keep-alive, HTTP/2, WebSocket, and proxy behavior before using Least Connections.
• Choose stable hash keys: avoid values that change between requests when affinity is required.
• Test hash distribution: evaluate realistic key populations rather than assuming uniform hashes produce uniform traffic.
• Monitor hot keys: detect tenants or resources generating disproportionate traffic under Consistent Hashing.
• Plan membership changes: test backend addition, removal, failure, and recovery behavior.
• Keep health separate from selection: route only among targets currently eligible to serve traffic.
• Measure per-target metrics: compare requests, active connections, latency, errors, CPU, memory, and saturation.
• Load-test realistic traffic: include slow requests, long-lived connections, skewed keys, and backend failures.
• Test reduced-capacity operation: confirm that remaining backends can absorb redistributed traffic after failures.
• Watch algorithm changes during deployments: ensure new targets do not receive inappropriate traffic before they are ready.
• Reevaluate as workloads evolve: an algorithm suitable for short stateless requests may become inappropriate after traffic patterns change.

Conclusion

Round Robin, Least Connections, and Consistent Hashing optimize different aspects of traffic routing. Round Robin provides simple and predictable distribution, Least Connections reacts to active connection counts, and Consistent Hashing provides stable placement for key-oriented workloads.

The correct choice depends on workload behavior rather than algorithm popularity. Request cost, connection lifetime, backend capacity, affinity requirements, protocol behavior, and failure scenarios should determine the routing strategy.

Key Takeaway

Choose a load-balancing algorithm based on the signal that best represents the routing problem. Use Round Robin for simple homogeneous traffic, Least Connections when connection distribution meaningfully reflects current work, and Consistent Hashing when stable key placement matters more than perfectly even request counts.

More Articles to Read

• Load Balancing Explained: Distributing Traffic at Scale
• Designing Highly Available Load Balancing Architectures
• Sticky Sessions and Stateless Applications
• Traffic Routing Strategies for Zero-Downtime Deployments
• Global Load Balancing and Multi-Region Traffic Routing
• Load Balancing Best Practices for Production Systems