What Is a Cache Stampede?

A cache stampede happens when many requests simultaneously discover that the same cached value is missing or expired and all attempt to regenerate it at once. Instead of one request rebuilding the value, hundreds or thousands may hit the database or another expensive dependency simultaneously.

The cache normally protects the backend from repeated work. During a stampede, that protection temporarily disappears exactly when traffic is high, potentially causing database saturation, increased latency, timeouts, cascading failures, and an outage.

Table of Contents

• How a Cache Stampede Happens
• Why Cache Stampedes Are Dangerous
• Cache Stampede vs Hot Key
• Preventing Cache Stampedes
  • Request Coalescing
  • Distributed Locking
  • Stale-While-Revalidate
  • TTL Jitter
  • Proactive Cache Refresh
• Choosing a Prevention Strategy
• Failure Scenarios
• Production Design Example
• Common Cache Stampede Mistakes
• Production Checklist
• Frequently Asked Questions
  • Is Every Cache Miss a Cache Stampede?
  • Does a Longer TTL Prevent Cache Stampedes?
  • Does Redis Automatically Prevent Cache Stampedes?
  • Is a Distributed Lock Always Required?
  • Is Serving Stale Data Safe?
• Conclusion
  • More Articles to Read

How a Cache Stampede Happens

Consider an API endpoint that serves a popular product page. The application caches the product data for 60 seconds so most requests avoid querying the database.

Request
   |
   v
Cache
   |
   +-- HIT --> Return cached value
   |
   +-- MISS --> Query database
                  |
                  v
               Cache value
                  |
                  v
               Response

Under normal traffic this works well. One request occasionally encounters a cache miss, loads the value from the database, stores it, and subsequent requests use the cached copy.

The problem appears when a popular key expires while many requests are arriving simultaneously.

Popular key expires
        |
        v
      MISS
   /   |   |   \
  /    |   |    \
 R1   R2  R3   R1000
 |     |    |      |
 v     v    v      v
 DB    DB   DB     DB

All requests observe the same missing key before any of them finishes rebuilding it.

If 5,000 requests arrive during that short window, the backend can receive thousands of queries for data that normally requires only one query per cache lifetime.

This behavior is also called a cache dogpile or thundering herd.

Why Cache Stampedes Are Dangerous

A cache stampede converts traffic that the cache normally absorbs into sudden backend load.

Suppose an endpoint receives 10,000 requests per second and has a 99.9% cache hit rate. The database may normally receive only a small fraction of that traffic.

When a highly requested key expires, the situation can temporarily become:

Normal

10,000 requests/sec
        |
        v
      Cache
        |
        +-- 9,990 cached
        |
        +-- 10 backend requests


Stampede

10,000 requests/sec
        |
        v
     Expired Key
        |
        v
Thousands of backend requests

The resulting load can exhaust:

• database connections;
• database CPU;
• application workers;
• network connections;
• external API quotas;
• thread pools;
• downstream service capacity.

Latency then increases. Slow requests remain active longer, consuming even more resources.

Cache expires
     |
     v
Backend load increases
     |
     v
Latency increases
     |
     v
Requests stay active longer
     |
     v
More resources consumed
     |
     v
Timeouts and retries
     |
     v
Even more load

A cache optimization can therefore become the trigger for a cascading failure.

Cache Stampede vs Hot Key

A cache stampede and a hot key are related but different problems.

A hot key can exist while the value remains cached successfully. The problem is that one cache partition receives disproportionate traffic.

A stampede happens when the protection for a popular value disappears and many callers attempt regeneration simultaneously.

Hot Key

Clients
  \ | /
   \|/
 Cache Key
    |
High cache traffic


Cache Stampede

Clients
  \ | /
   \|/
Missing Key
  / | \
 /  |  \
DB  DB  DB

The problems can also reinforce each other. A heavily requested hot key is particularly dangerous when it expires because its request volume can produce a large stampede.

For a broader treatment of both failure modes, see Preventing Cache Stampedes and Hot Keys.

Preventing Cache Stampedes

The central goal is simple: avoid allowing every request that sees a cache miss to regenerate the same value independently.

Several techniques solve different versions of the problem. Production systems often combine them rather than relying on a single mechanism.

Request Coalescing

Request coalescing allows one request to perform the expensive work while concurrent requests wait for the same result.

R1 ----+
R2 ----+
R3 ----+----> One regeneration ----> Database
R4 ----+              |
R5 ----+              v
                    Result
                      |
              +-------+-------+
              v       v       v
             R1      R2      R3...

Within a single application process, this can be implemented with an in-memory map of work currently in progress.

import asyncio


in_flight: dict[str, asyncio.Task] = {}


async def get_or_load(key: str):
    cached = await cache.get(key)

    if cached is not None:
        return cached

    task = in_flight.get(key)

    if task is None:
        task = asyncio.create_task(load_and_cache(key))
        in_flight[key] = task

    try:
        return await task
    finally:
        if in_flight.get(key) is task:
            in_flight.pop(key, None)

This prevents duplicate regeneration inside one process.

It does not coordinate separate application instances:

Instance A -- one regeneration
Instance B -- one regeneration
Instance C -- one regeneration
Instance D -- one regeneration

For a horizontally scaled application, another coordination mechanism may be needed when even one regeneration per instance is too expensive.

Distributed Locking

A distributed lock allows one application instance to become the cache regenerator for a key.

Cache MISS
    |
    v
Acquire lock for product:481
    |
 +--+----------------+
 |                   |
success             failure
 |                   |
 v                   v
Load backend      Wait / retry /
 |                serve stale
 v
Update cache
 |
 v
Release lock

A Redis-style lock might use an atomic conditional write:

lock_key = f"lock:{cache_key}"

acquired = await redis.set(
    lock_key,
    request_id,
    nx=True,
    ex=5,
)

if acquired:
    value = await load_from_database()
    await redis.set(cache_key, value, ex=60)
    await release_lock_safely(lock_key, request_id)

The lock needs an expiration so a crashed owner does not leave the key locked indefinitely.

Lock release should also verify ownership. One process should not delete a lock that expired and was subsequently acquired by another process.

Distributed locking introduces its own failure modes, so it should be used when the protected regeneration work justifies the coordination cost. The mechanics and trade-offs are covered in What Is a Distributed Lock?.

Stale-While-Revalidate

Another strategy is to keep an expired value available for a limited period while one worker refreshes it.

Fresh
  |
  | TTL expires
  v
Stale but usable
  |
  +-- requests receive stale value
  |
  +-- one refresh starts
          |
          v
       Backend
          |
          v
     New cached value

Instead of making every request wait for regeneration, the system returns slightly stale data immediately.

A cache entry can conceptually contain two expiration times:

{
  "value": {
    "product_id": 481,
    "price": "49.99"
  },
  "fresh_until": 1791302400,
  "stale_until": 1791302460
}

Before fresh_until, the value is served normally. Between fresh_until and stale_until, the old value may still be served while refresh happens asynchronously. After the stale deadline, the application may require synchronous regeneration.

This technique is particularly useful when slightly stale data is acceptable and low response latency is more important than immediate freshness.

TTL Jitter

A different stampede can occur when many cache entries are created together with the same TTL.

Suppose 100,000 keys are populated during a deployment:

12:00:00
100,000 keys cached
TTL = 10 minutes

12:10:00
100,000 keys expire together

The backend can receive a large synchronized wave of misses.

TTL jitter randomizes expiration times:

import random


BASE_TTL = 600
JITTER = 120

ttl = BASE_TTL + random.randint(
    -JITTER,
    JITTER,
)

await cache.set(
    key,
    value,
    ttl=ttl,
)

Now expiration is spread across a window rather than concentrated at one instant.

Without jitter

----------|----------------
          ^
     many expirations


With jitter

-------|-|--|-||-|---------
       ^ ^  ^ ^^ ^
   distributed expirations

Jitter does not prevent multiple requests from racing on a single extremely popular key. It reduces synchronized expiration across many keys.

Proactive Cache Refresh

Frequently accessed values can be refreshed before their TTL expires.

Cached
  |
  | 80% of TTL elapsed
  v
Background refresh
  |
  v
New value stored
  |
  v
Original expiration never becomes a miss

This can work well for predictable hot data such as configuration, catalog metadata, popular rankings, or expensive aggregates.

The trade-off is additional background work. Refreshing every cached value proactively can waste resources on keys that are no longer being requested.

A better policy often refreshes only values with sufficient recent request frequency.

Choosing a Prevention Strategy

No single cache-stampede technique is optimal for every workload.

A high-traffic system may combine several strategies:

TTL jitter
    |
    v
Fewer synchronized expirations
    |
    v
Stale-while-revalidate
    |
    v
Requests continue receiving data
    |
    v
Distributed refresh lock
    |
    v
Only one backend regeneration

The correct combination depends on traffic shape, regeneration cost, freshness requirements, cache topology, and acceptable failure behavior.

Failure Scenarios

Stampede protection needs explicit behavior when the cache or coordination mechanism fails.

Cache unavailable. Sending every request directly to the database may create an immediate overload. Applications may need concurrency limits, load shedding, or degraded responses to protect the backend.

Lock owner crashes. A distributed lock needs bounded lifetime. Otherwise every request can remain blocked behind a lock whose owner no longer exists.

Regeneration fails. Waiting requests should not immediately launch another synchronized wave of regeneration. Backoff, stale data, or a short negative/error cache may be appropriate depending on the failure.

Database becomes slow. Longer regeneration increases the time during which requests accumulate. This makes stampede protection even more important during dependency degradation.

Popular value does not exist. Repeated requests for the same nonexistent resource can create a similar backend amplification problem.

GET /products/does-not-exist
        |
        v
Cache MISS
        |
        v
Database MISS
        |
        v
Nothing cached
        |
        v
Next request repeats everything

A short-lived negative cache can reduce repeated lookups when application semantics permit it.

The cache should also be treated as a dependency with its own reliability behavior. Caching Best Practices for Production Systems covers the broader operational considerations.

Production Design Example

Consider a product API receiving heavy traffic during a major sale.

Clients
   |
   v
Load Balancer
   |
   +---------+---------+
   |         |         |
   v         v         v
 API-1     API-2     API-3
   \         |         /
    \        |        /
     +-------+-------+
             |
             v
           Redis
             |
             v
         PostgreSQL

The endpoint is:

GET /products/481

Product 481 becomes extremely popular and receives 8,000 requests per second.

The cached representation has a base TTL of five minutes. Without protection, expiration can cause thousands of API requests to query PostgreSQL simultaneously.

A safer design uses several layers.

Normal request.

Request
   |
   v
Redis
   |
   +-- fresh --> return immediately

Soft expiration.

Request
   |
   v
Redis
   |
   +-- stale but usable
          |
          +--> return stale value
          |
          +--> attempt refresh lock

Refresh winner.

Acquire refresh lock
        |
        v
Query PostgreSQL
        |
        v
Update Redis
        |
        v
Release lock

Refresh loser.

Refresh lock already held
        |
        v
Return stale cached value

Only one application instance regenerates the value while the remaining requests continue receiving the previous version.

The cache TTL also receives random jitter so unrelated product entries do not expire simultaneously.

A simplified Python flow could look like:

async def get_product(product_id: int):
    key = f"product:{product_id}"

    entry = await cache.get(key)

    if entry and entry.is_fresh():
        return entry.value

    if entry and entry.is_stale_but_usable():
        acquired = await try_refresh_lock(key)

        if acquired:
            schedule_refresh(product_id, key)

        return entry.value

    acquired = await try_refresh_lock(key)

    if acquired:
        return await refresh_product(
            product_id,
            key,
        )

    return await wait_for_refreshed_value(key)

The exact waiting behavior needs a timeout. A request should not wait indefinitely because another worker owns a refresh lock.

Useful production metrics include:

The important relationship is not only cache hit rate. A system can have a 99% hit rate and still overload the database if the remaining 1% of requests arrive simultaneously for expensive keys.

Common Cache Stampede Mistakes

• Assuming TTL alone is sufficient. Expiration creates the miss that can trigger a stampede.
• Using identical TTLs for large groups of keys. Synchronized expiration can create a large backend traffic spike.
• Locking without expiration. A crashed lock owner can block regeneration indefinitely.
• Releasing locks without ownership checks. One worker can accidentally remove another worker's newer lock.
• Holding distributed locks longer than necessary. Slow lock holders increase contention and latency.
• Allowing unlimited waiters. Thousands of blocked requests can still exhaust application resources.
• Retrying regeneration immediately. Failed regeneration can create another thundering herd.
• Sending all traffic to the database when the cache fails. Cache failure can become database failure.
• Refreshing every key proactively. Background refresh can consume more resources than it saves.
• Ignoring nonexistent values. Repeated misses for missing resources can repeatedly hit the backend.
• Monitoring only cache hit rate. Hit rate can hide concentrated regeneration of hot keys.

Production Checklist

• Identify high-traffic cache keys.
• Measure the cost of regenerating each important cached value.
• Define whether stale values are acceptable.
• Coalesce concurrent regeneration where possible.
• Use distributed coordination when regeneration must be limited across instances.
• Give distributed locks bounded expiration.
• Verify lock ownership before releasing locks.
• Bound how long requests wait for another refresh.
• Add jitter when many keys use similar TTLs.
• Consider stale-while-revalidate for read-heavy workloads.
• Consider proactive refresh for predictable hot data.
• Use negative caching where repeated missing lookups are expensive and semantics allow it.
• Protect databases with connection and concurrency limits.
• Define behavior when the cache is unavailable.
• Avoid unlimited fallback traffic to the database.
• Monitor cache misses by key or key group.
• Monitor regeneration rate.
• Monitor refresh-lock contention.
• Monitor stale responses.
• Monitor backend load during cache expiration.
• Test popular-key expiration under realistic concurrency.
• Test cache outages.
• Test slow regeneration.
• Test lock-owner crashes.

Frequently Asked Questions

Cache stampedes are primarily a concurrency problem. The cache miss itself is normal; the dangerous behavior is many callers performing the same expensive recovery work simultaneously.

Is Every Cache Miss a Cache Stampede?

No. A normal cache miss may result in one request loading a value and caching it.

A stampede occurs when many concurrent requests encounter the same missing or expired value and independently regenerate it, producing amplified backend traffic.

Does a Longer TTL Prevent Cache Stampedes?

A longer TTL makes expiration less frequent but does not remove the concurrency problem. When a sufficiently popular key eventually expires, many requests can still regenerate it simultaneously.

TTL duration should be chosen primarily from freshness and caching requirements, while stampede protection should explicitly control regeneration.

Does Redis Automatically Prevent Cache Stampedes?

No. Redis can provide primitives useful for preventing stampedes, such as atomic conditional writes for locks, but an ordinary cache-aside implementation can still allow every application instance to query the backend after a miss.

The application architecture must define how regeneration is coordinated.

Is a Distributed Lock Always Required?

No. Request coalescing may be enough for a single process, while stale-while-revalidate can make strict locking unnecessary for some workloads.

A distributed lock is useful when regeneration is expensive enough that concurrent refreshes across application instances must be controlled.

Is Serving Stale Data Safe?

It depends on the data. Serving a product description that is several seconds old may be acceptable, while stale authorization state, inventory guarantees, or financial data may not be.

Freshness policy should be based on business semantics rather than applied uniformly to every cache entry.

Conclusion

A cache stampede occurs when many requests simultaneously regenerate the same missing or expired cached value. The resulting traffic amplification can overload the database or another backend dependency even when the cache normally absorbs most application traffic.

Request coalescing, distributed locking, stale-while-revalidate, TTL jitter, and proactive refresh address different parts of the problem. Production systems often combine several of these techniques based on traffic patterns, regeneration cost, freshness requirements, and acceptable failure behavior.

Key Takeaway: a cache miss should not automatically give every concurrent request permission to perform the same expensive backend work. Coordinate regeneration, spread expiration, serve stale data where appropriate, and protect backend capacity when cache protection temporarily disappears.

More Articles to Read

• Caching Strategies
• What Is Cache Invalidation?
• Caching Best Practices for Distributed Applications