Scaling Elasticsearch Clusters

Elasticsearch can scale from a small single-node deployment to clusters serving billions of documents, but horizontal scaling is not automatic. Adding nodes without understanding shard layout, workload distribution, recovery behavior, and memory pressure can make a cluster more complex without making it faster.

A scalable Elasticsearch architecture starts with a simple idea: distribute data and workload in a way that keeps every node within safe CPU, memory, disk, and network limits.

This article focuses on the practical engineering decisions that matter most when an Elasticsearch cluster grows: shards, replicas, node roles, hot spots, routing, recovery, capacity planning, and the operational signals that indicate when scaling is actually needed.

Table of Contents

• What Scaling Elasticsearch Really Means
• Shards Are the Unit of Scale
  • Primary Shards
  • Replica Shards
  • Choosing Sustainable Shard Sizes
• Separate Node Responsibilities
• Scale Search Throughput
• Scale Indexing Throughput
• Prevent Hot Shards and Hot Nodes
• Plan for Rebalancing and Recovery
• Capacity Planning for Growth
• Production Design Example
• Common Mistakes
  • Creating Too Many Shards
  • Creating Too Few Shards
  • Using Replicas to Solve the Wrong Problem
  • Running the Cluster Too Hot
• Production Checklist
• Conclusion

What Scaling Elasticsearch Really Means

Scaling Elasticsearch means solving one or more different bottlenecks:

Need more storage?
Need more search throughput?
Need more indexing throughput?
Need lower latency?
Need better failure tolerance?
Need faster recovery?

These are different problems and may require different changes.

For example, adding replica shards can improve search concurrency, but replicas do not increase the amount of primary data the cluster can store. Adding more data nodes increases storage and compute capacity, but may not help if all queries are routed to one hot shard.

A useful mental model is:

Cluster Capacity
    |
    +--> CPU
    +--> heap
    +--> page cache
    +--> disk capacity
    +--> disk throughput
    +--> network bandwidth
    +--> shard distribution

Scaling succeeds only when the limiting resource is identified correctly.

Shards Are the Unit of Scale

An Elasticsearch index is divided into shards. Each shard is effectively an independent Lucene index that can be placed on a node and queried in parallel.

Index: products

+----------+----------+----------+
| Shard 0  | Shard 1  | Shard 2  |
+----------+----------+----------+

Sharding determines how Elasticsearch distributes storage and search work across a cluster.

Primary Shards

Primary shards divide the authoritative contents of an index.

Suppose a 900 GB index has three primary shards:

900 GB index
    |
    +--> Primary 0: ~300 GB
    +--> Primary 1: ~300 GB
    +--> Primary 2: ~300 GB

If the cluster later grows to 20 data nodes, that index still has only three primary shard partitions. The cluster cannot distribute those three primary shards across twenty nodes for indexing work.

This is why the initial shard strategy affects future scaling.

However, creating hundreds of shards in anticipation of future growth is usually worse. Every shard has overhead, and every search touching the full index may fan out across all of them.

Replica Shards

A replica is a copy of a primary shard.

Shard 0
   |
   +--> Primary
   +--> Replica

Shard 1
   |
   +--> Primary
   +--> Replica

Replicas primarily provide two benefits:

• availability when a node or primary shard fails;
• additional capacity for search requests.

With one replica:

Primary Shard 0 -> Node A
Replica Shard 0 -> Node B

Search requests can be served by either copy.

However, every indexed document must also be replicated:

Index request
     |
     v
Primary
     |
     v
Replica

Increasing replica count therefore increases read capacity and redundancy but also increases disk usage and write replication work.

Choosing Sustainable Shard Sizes

There is no universal shard size that works for every cluster. The right size depends on document structure, query patterns, update rate, hardware, and recovery objectives.

The important trade-off is between large shards and many small shards.

Large shards
    |
    +--> fewer shard operations
    +--> less metadata overhead
    |
    +--> slower relocation
    +--> slower recovery
    +--> less placement flexibility


Many small shards
    |
    +--> more placement flexibility
    +--> more parallelism
    |
    +--> more heap overhead
    +--> more query fan-out
    +--> more cluster metadata

A practical design should target shard sizes that are large enough to avoid oversharding but small enough to move and recover within acceptable operational windows.

Shard sizing should be validated with production-like workloads rather than chosen from a fixed rule.

Separate Node Responsibilities

Small Elasticsearch clusters often let every node perform several responsibilities. That is convenient initially, but larger clusters benefit from clearer workload separation.

A simplified architecture might contain:

                 Cluster

        +----------------------+
        | Cluster Manager Nodes|
        +----------------------+

      +-----------+   +-----------+
      | Data Node |   | Data Node |
      +-----------+   +-----------+

      +-----------+   +-----------+
      | Data Node |   | Data Node |
      +-----------+   +-----------+

             +---------------+
             | Coordinating  |
             | Nodes         |
             +---------------+

Data nodes store shards and execute search and indexing work.

Cluster-manager-eligible nodes maintain cluster coordination and metadata. Protecting these nodes from heavy data workloads helps reduce the risk that search or indexing pressure affects cluster stability.

Coordinating nodes can receive client requests and merge shard-level results:

Client
  |
  v
Coordinating Node
  |
  +--> Shard on Node A
  +--> Shard on Node B
  +--> Shard on Node C
  |
  v
Merge
  |
  v
Response

Separating responsibilities becomes more useful as cluster size and traffic increase, but dedicated roles also introduce infrastructure cost. A small cluster does not need every possible node role separated from day one.

Scale Search Throughput

Search requests are distributed across shards. Each shard performs local retrieval and scoring, then a coordinating node combines the results.

Search Request
      |
      v
Coordinator
  /    |    \
 v     v     v
S0    S1    S2
 \     |     /
  \    |    /
   v   v   v
 Merge Results

Search throughput can often be increased by adding replicas because more shard copies become available to serve queries.

For example:

3 primary shards
1 replica each

Total shard copies:
6

Queries can be distributed
across more nodes.

But replicas are useful only when the cluster has enough nodes to place them separately and enough CPU, memory, and disk throughput to use the extra copies.

If the workload is CPU-bound, adding data nodes and distributing shard copies across them can increase capacity.

If the workload is dominated by expensive queries, infrastructure scaling alone may be inefficient. Query optimization should come first.

Common causes of expensive searches include:

large aggregations
deep pagination
broad wildcard queries
high-cardinality sorting
script scoring
large result windows
queries touching too many shards

Search architecture should therefore combine cluster scaling with bounded query design. The broader performance patterns are discussed in Designing High-Performance Search Systems.

Scale Indexing Throughput

Indexing behaves differently from search because writes first target the primary shard for a document.

Document
   |
   v
Routing
   |
   v
Primary Shard
   |
   +--> Replica 1
   +--> Replica 2

Adding replicas does not increase primary indexing parallelism. It actually creates additional replication work.

Indexing throughput is more directly affected by:

• number of primary shards;
• number of data nodes;
• bulk request sizing;
• refresh frequency;
• replica count;
• document complexity;
• storage throughput;
• merge activity.

Bulk operations typically provide higher throughput than individual document requests:

Individual writes:

doc -> request
doc -> request
doc -> request
doc -> request


Bulk:

[doc, doc, doc, doc, ...]
          |
          v
      bulk request

Batch size should remain bounded. Larger batches reduce per-request overhead but can increase memory consumption, latency, and retry cost when failures occur.

Refresh frequency is another important trade-off.

Frequent refresh
    |
    +--> low search visibility delay
    +--> more refresh overhead

Less frequent refresh
    |
    +--> higher indexing throughput
    +--> longer visibility delay

For large bulk imports, temporary changes to refresh behavior or replica strategy can improve throughput, provided the durability and freshness implications are understood.

Prevent Hot Shards and Hot Nodes

A cluster can have plenty of total capacity and still perform poorly if traffic is unevenly distributed.

Consider six nodes:

Node A: CPU 92%
Node B: CPU 88%
Node C: CPU 31%
Node D: CPU 28%
Node E: CPU 25%
Node F: CPU 29%

This is not primarily a cluster-capacity problem. It is a distribution problem.

Hot spots often appear when routing keys have uneven cardinality.

For example:

tenant_id = 1
40% of all traffic

tenant_id = 2
20%

remaining 50,000 tenants
40%

If documents are routed entirely by tenant, one shard may receive a disproportionate amount of traffic.

Shard A -> tenant 1 -> HOT
Shard B -> tenants 2-10000
Shard C -> tenants 10001-20000
...

The same problem can happen with time-based workloads when nearly every request targets the newest index.

2026-06 -> cold
2026-07 -> cold
2026-08 -> warm
2026-09 -> HOT

Monitoring should therefore inspect node-level and shard-level metrics, not only cluster averages.

Useful indicators include:

CPU per node
heap per node
disk I/O per node
search rate per shard
indexing rate per shard
query latency per shard
thread-pool queues
rejections

Adding nodes will not reliably solve a skewed routing strategy unless shard allocation and request distribution actually use the additional capacity.

Plan for Rebalancing and Recovery

Scaling a cluster often means moving large amounts of data.

When a node is added:

Before:

Node A -> S0 S3
Node B -> S1 S4
Node C -> S2 S5


Add Node D

        |
        v

Cluster redistributes shards

Shard relocation consumes:

• disk reads;
• disk writes;
• network bandwidth;
• CPU;
• cluster coordination work.

Scaling itself can therefore temporarily increase load.

The same is true during failure recovery:

Node B fails
    |
    v
replicas promoted
    |
    v
missing replicas recreated
    |
    v
large shard transfers

If a cluster normally runs near saturation, recovery traffic may push the remaining nodes beyond safe limits.

This is why production clusters need capacity headroom.

Normal load:
60-70%

Failure:
extra searches
+ shard recovery
+ replica recreation

Still within capacity

Recovery time should also influence shard size. A shard that is easy to store may still be operationally problematic if moving it takes hours.

Capacity Planning for Growth

Capacity planning should start from observed workload characteristics rather than from document count alone.

Useful inputs include:

primary index size
daily data growth
replica count
peak query rate
peak indexing rate
average document size
p95 / p99 latency
CPU utilization
heap utilization
disk utilization
storage throughput
network throughput

Suppose a cluster currently has:

primary data:       4 TB
replicas:           1
physical index data: ~8 TB

monthly growth:     600 GB primary
peak search:        7,000 QPS
peak indexing:      15,000 docs/sec

data nodes:         10
disk/node:          2 TB

Raw storage math alone would suggest:

10 nodes * 2 TB
= 20 TB raw capacity

But production capacity cannot consume all 20 TB. Space is needed for growth, shard relocation, merges, recovery, and operational safety.

A better model includes headroom:

Raw capacity
    -
failure reserve
    -
growth reserve
    -
merge / operational headroom
    =
usable capacity

Capacity should be tested under failure scenarios as well as normal traffic.

A useful benchmark asks:

Can the cluster meet p99 latency
while one data node is unavailable
and normal indexing continues?

If the answer is no, the cluster may already be effectively undersized even though it operates normally most of the time.

Production Design Example

Consider an Elasticsearch cluster supporting product search across 200 million documents.

The workload has:

primary index size:   6 TB
replicas:             1

peak search rate:     12,000 QPS
peak indexing rate:   25,000 docs/sec

search p95 target:    120 ms
search p99 target:    250 ms

The cluster could be structured as:

                    Clients
                       |
                       v
                Load Balancer
                       |
             +---------+---------+
             |                   |
             v                   v
       Coordinator A       Coordinator B
             |                   |
             +---------+---------+
                       |
        +--------------+--------------+
        |              |              |
        v              v              v
    Data Node       Data Node      Data Node
       ...             ...            ...
        |              |              |
        +--------------+--------------+

        Dedicated Cluster Manager Nodes

The index is divided into enough primary shards to distribute data and indexing work across the data tier without creating excessive fan-out.

Each primary has one replica:

Primary 0 -> Node A
Replica 0 -> Node F

Primary 1 -> Node B
Replica 1 -> Node G

Primary 2 -> Node C
Replica 2 -> Node H

A search request enters through a coordinating node:

Query
  |
  v
Coordinator
  |
  +--> shard copy
  +--> shard copy
  +--> shard copy
  +--> shard copy
  |
  v
Top candidates merged
  |
  v
Response

Product updates arrive through an indexing pipeline:

Product Services
      |
      v
Event Stream
      |
      v
Indexing Workers
      |
      | bounded bulk requests
      v
Elasticsearch

The cluster is sized so that normal traffic consumes only part of available capacity:

Normal:
CPU ~55-65%
heap within safe range
disk well below high watermark
minimal queueing

Node failure:
CPU rises
replicas promoted
recovery begins

Cluster still serves traffic
within degraded but acceptable SLO

Autoscaling or planned expansion should be triggered before sustained saturation rather than after latency collapses.

Operational alerts might watch for:

sustained CPU pressure
heap pressure
search queue growth
rejected operations
disk watermark approach
uneven shard allocation
indexing lag
rapid storage growth
p99 latency regression

The important principle is that scaling decisions are driven by resource pressure and workload behavior, not only by total document count.

Common Mistakes

Creating Too Many Shards

Oversharding is one of the easiest ways to make a large Elasticsearch cluster inefficient.

Suppose a 200 GB index is divided into 200 primary shards:

200 GB
  |
  v
200 primary shards

~1 GB per shard

A search touching the whole index may now require hundreds of shard-level operations.

1 query
   |
   v
200 shard searches
   |
   v
200 partial results
   |
   v
merge

Each shard also consumes metadata and memory and increases the work involved in cluster coordination.

More shards create more parallelism only up to the point where coordination overhead dominates.

Creating Too Few Shards

The opposite extreme limits distribution.

Consider a rapidly growing 4 TB index with only two primary shards:

4 TB
 |
 +--> 2 TB shard
 +--> 2 TB shard

Even a large cluster cannot divide those primary partitions across many nodes.

Large shards also take longer to recover or move after failures.

The correct shard strategy must account for expected growth before individual shards become operationally difficult to manage.

Using Replicas to Solve the Wrong Problem

Increasing replicas can improve read capacity:

1 primary
1 replica
    |
    v
2 searchable copies

But replicas do not solve every bottleneck.

If indexing throughput is the problem:

Primary receives write
       |
       +--> Replica
       +--> Replica
       +--> Replica

More replicas create more replication work.

If storage capacity is the problem, replicas also consume additional storage rather than creating more usable primary capacity.

Replica count should be chosen for availability and search throughput requirements, not used as a generic scaling control.

Running the Cluster Too Hot

A cluster operating at 90-95% resource utilization may appear cost-efficient until a node fails.

Normal:
10 nodes at 92%

One node fails:
remaining 9 nodes
+ redistributed search traffic
+ shard recovery
+ replica recreation

The failure creates additional work precisely when available capacity has decreased.

Production clusters should maintain enough headroom to absorb traffic spikes, rolling upgrades, rebalancing, and node failures without cascading overload.

Production Checklist

• Identify whether the bottleneck is storage, search, indexing, or recovery before scaling.
• Plan primary shard count around expected growth and workload distribution.
• Avoid both oversized shards and excessive shard counts.
• Use replicas for availability and additional search capacity.
• Do not expect replicas to increase primary indexing throughput.
• Separate node responsibilities when cluster size justifies it.
• Monitor shard-level hot spots rather than cluster averages alone.
• Use routing only when the key distributes workload safely.
• Use bounded bulk indexing and monitor rejections.
• Balance refresh frequency against freshness requirements.
• Keep disk, CPU, memory, and network headroom for recovery.
• Measure shard relocation and recovery times.
• Capacity-test with realistic search and indexing traffic together.
• Test node-failure scenarios before production growth makes them urgent.
• Scale before sustained resource saturation causes tail-latency collapse.

Conclusion

Elasticsearch scales by distributing shards and workload across nodes, but good scaling depends on much more than adding hardware. Primary shards determine how data and indexing work can be partitioned, replicas provide redundancy and additional search capacity, and node placement determines whether that capacity is actually usable.

The most important production problems usually come from imbalance: too many shards, shards that are too large, uneven routing, hot nodes, insufficient recovery headroom, or clusters that remain close to saturation during normal operation.

Capacity planning should therefore combine index growth, query throughput, indexing rate, shard distribution, recovery time, and failure scenarios rather than relying on document count or storage alone.

Key Takeaway: Elasticsearch scaling works best when shards remain manageable, workload is evenly distributed, replicas are used for the problems they actually solve, and enough spare capacity exists to survive failures and rebalancing without losing latency targets.

More Articles to Read

• Search Engines Explained: How Modern Search Works
• Designing High-Performance Search Systems
• Search Relevance and Ranking Strategies
• Search Best Practices for Production Systems