Designing Multi-Level Caching Architectures

Large production systems rarely rely on a single cache. Modern architectures typically combine browser caches, CDNs, reverse proxies, local in-memory caches, distributed caches, and database caches. Each layer removes a different bottleneck and serves requests at a different point in the request path.

Designing an effective multi-level cache is not about adding more caches. Every additional layer introduces synchronization, invalidation, observability, and failure complexity. The goal is to eliminate the most expensive work while keeping stale data predictable and operational behavior manageable.

Table of Contents

• Why Multi-Level Caching
• Cache Layers
  • Browser Cache (L0)
  • CDN (L1)
  • Reverse Proxy (L2)
  • Application Cache (L3)
• Cache Request Flow
  • Read Flow
  • Write Flow
• Python Example
• Production Design Example
  • Architecture
  • Failure Scenarios
  • Monitoring
• Common Mistakes
• Production Checklist
• Conclusion
• More Articles to Read

Why Multi-Level Caching

No single cache can optimize every request.

A CDN is excellent for globally shared static content but cannot understand business rules. An application cache understands tenants, permissions, and entities but every request still reaches the application. Local memory is extremely fast but cannot be shared across servers.

Each layer removes a different amount of work.

User
 |
 | Browser cache
 |
 v
CDN
 |
 v
Reverse Proxy
 |
 v
Application
 |
 +------ Local Memory Cache
 |
 +------ Redis
 |
 v
Database

The higher a request is satisfied, the more infrastructure is bypassed.

The objective is to maximize requests served by the earliest safe cache while keeping invalidation manageable.

Cache Layers

Production systems usually combine several cache layers instead of relying on a single technology.

Browser Cache (L0)

The browser is the fastest cache because it eliminates the network entirely.

• CSS
• JavaScript
• Fonts
• Images
• Downloaded documents

Versioned asset filenames allow one-year immutable caching.

from fastapi import Response


def cache_static(response: Response) -> None:
    response.headers["Cache-Control"] = (
        "public, max-age=31536000, immutable"
    )

CDN (L1)

The CDN serves globally shared responses before traffic reaches the origin.

Typical candidates include:

• public articles
• documentation
• product images
• public APIs
• video thumbnails

Because every cache miss still reaches the origin, cache hit ratio directly affects backend load.

Reverse Proxy (L2)

The reverse proxy sits inside the origin infrastructure and shields application servers.

Unlike the CDN, it is usually deployed in one or a few regions.

Besides caching, it often provides:

• load balancing
• TLS termination
• compression
• rate limiting
• health checks

Reverse proxies are especially valuable when multiple CDN edge locations generate simultaneous cache misses.

Application Cache (L3)

The application cache understands business objects.

Unlike HTTP caches, it can safely cache:

• tenant configuration
• permissions
• database queries
• shipping rates
• pricing
• authentication metadata
• feature flags

Redis is commonly used because it is shared by every application instance.

Local memory caches are even faster but require synchronization between servers.

Cache Request Flow

A request moves through cache layers until one returns a valid response.

Read Flow

User
 |
Browser cache?
 |
 +-- HIT --> Return
 |
 +-- MISS
 |
CDN?
 |
 +-- HIT --> Return
 |
 +-- MISS
 |
Reverse Proxy?
 |
 +-- HIT --> Return
 |
 +-- MISS
 |
Application Memory?
 |
 +-- HIT --> Return
 |
 +-- MISS
 |
Redis?
 |
 +-- HIT --> Return
 |
 +-- MISS
 |
Database

Each successful cache hit eliminates the work performed by every lower layer.

Write Flow

Writes are more complicated because cached copies must eventually reflect committed data.

Application
      |
Database Commit
      |
Outbox Event
      |
Redis Invalidation
      |
Reverse Proxy Purge
      |
CDN Purge
      |
Browser receives updated asset/version

Notice that invalidation always moves outward from the authoritative database.

Updating caches before the database commits risks exposing data that never became durable.

Python Example

The following service combines a local in-memory cache with Redis.

Most repeated requests never reach Redis, while cache misses still avoid database queries.

from __future__ import annotations

import asyncio
import json
from dataclasses import asdict, dataclass
from datetime import datetime, timedelta

from cachetools import TTLCache
from redis.asyncio import Redis


@dataclass(frozen=True)
class Product:
    product_id: str
    title: str
    price: float
    version: int


LOCAL_CACHE = TTLCache(
    maxsize=5000,
    ttl=15,
)


class ProductCache:

    def __init__(
        self,
        redis: Redis,
        repository,
    ):
        self.redis = redis
        self.repository = repository
        self.locks: dict[str, asyncio.Lock] = {}

    async def get(self, product_id: str):

        if product_id in LOCAL_CACHE:
            return LOCAL_CACHE[product_id]

        cached = await self.redis.get(product_id)

        if cached:

            product = Product(
                **json.loads(cached)
            )

            LOCAL_CACHE[product_id] = product

            return product

        lock = self.locks.setdefault(
            product_id,
            asyncio.Lock(),
        )

        async with lock:

            if product_id in LOCAL_CACHE:
                return LOCAL_CACHE[product_id]

            product = await self.repository.get(
                product_id
            )

            if product is None:
                return None

            await self.redis.set(
                product_id,
                json.dumps(asdict(product)),
                ex=300,
            )

            LOCAL_CACHE[product_id] = product

            return product

This architecture provides:

• L3 local-memory cache hit in microseconds.
• L4 Redis hit in sub-millisecond latency.
• Database access only when every cache misses.
• Per-key locking to prevent cache stampedes inside one application instance.

Production Design Example

Consider a global product catalog used by public storefronts, authenticated account dashboards, internal services, and pricing workflows. The catalog contains mostly stable product metadata, while availability, account-specific pricing, and permissions change more frequently.

A single cache policy would either serve stale business data or generate unnecessary backend traffic. The architecture therefore assigns each response to the highest cache layer that can safely serve it.

Architecture

Global users
    |
    v
Browser cache
    |
    v
CDN
    |
    +---- static assets
    +---- public product pages
    |
    v
Reverse proxy
    |
    +---- shared public API responses
    +---- origin request coalescing
    |
    v
Application service
    |
    +---- local in-memory cache
    |       |
    |       +---- feature metadata
    |       +---- product summaries
    |
    +---- Redis distributed cache
    |       |
    |       +---- product details
    |       +---- tenant configuration
    |       +---- pricing results
    |
    v
PostgreSQL and external pricing providers

The layers have separate responsibilities:

• Browser cache: immutable JavaScript, CSS, fonts, and images.
• CDN: public assets and anonymous product pages.
• Reverse proxy: shared API responses and protection against simultaneous CDN misses.
• Local memory: tiny frequently reused values with very short TTLs.
• Redis: shared domain objects and expensive calculated responses.
• PostgreSQL: authoritative product, account, and pricing configuration.

Authenticated account pricing bypasses shared HTTP caches because the response depends on account contracts, permissions, currency, quantity, and pricing version.

from __future__ import annotations

import hashlib
from dataclasses import dataclass
from decimal import Decimal


@dataclass(frozen=True)
class PricingCacheKey:
    account_id: int
    product_id: str
    quantity: int
    currency: str
    pricing_version: int

    def build(self) -> str:
        source = (
            f"{self.account_id}:"
            f"{self.product_id}:"
            f"{self.quantity}:"
            f"{self.currency}:"
            f"{self.pricing_version}"
        )

        digest = hashlib.sha256(
            source.encode("utf-8")
        ).hexdigest()

        return f"pricing:v1:{digest}"


@dataclass(frozen=True)
class CalculatedPrice:
    product_id: str
    currency: str
    unit_price: Decimal
    total_price: Decimal
    pricing_version: int

The key includes every business input that can change the result. Using only the product ID would incorrectly share one account’s negotiated price with another account.

Failure Scenarios

The local cache contains an old value. Local TTLs remain short, and invalidation events clear entries from every application instance where practical. Redis or the database remains the fallback source.

Redis becomes unavailable. Local hits continue temporarily. Misses fall back to PostgreSQL through bounded connection pools and request concurrency limits. Noncritical expensive calculations may be rejected rather than overloading the database.

The CDN becomes cold after deployment. The reverse proxy consolidates repeated edge misses so application instances do not execute the same request for every edge location.

The reverse proxy restarts and loses its cache. CDN hits continue serving most public traffic. Proxy request locking prevents concurrent misses from immediately overwhelming the application.

An invalidation event is delayed. Local and Redis TTLs bound stale duration. Public HTTP responses use short shared TTLs or versioned URLs where immediate replacement matters.

A database update commits but cache invalidation fails. The outbox event remains pending and is retried. Finite TTLs provide a final recovery mechanism if retries continue failing.

A popular key expires simultaneously across all layers. TTL jitter, stale-while-revalidate, reverse-proxy locking, and application-level request coalescing prevent every request from reaching the database.

A cache node reaches its memory limit. Redis begins evicting keys, reducing hit ratio and increasing database traffic. Alerts must trigger before eviction becomes sustained.

Monitoring

Each cache layer should expose its own hit ratio, latency, capacity, and fallback behavior. A global cache hit ratio is insufficient because it can hide an ineffective or failing layer.

Useful end-to-end signals include:

• which layer served the request
• total request latency
• cache value age
• invalidation delay
• database calls caused by cache misses
• requests rejected during degraded cache operation
• cold-cache recovery duration

A small result object can make application cache behavior visible to metrics and tracing:

from __future__ import annotations

from dataclasses import dataclass
from enum import Enum
from typing import Generic, TypeVar


ValueT = TypeVar("ValueT")


class CacheSource(str, Enum):
    LOCAL = "local"
    REDIS = "redis"
    DATABASE = "database"


@dataclass(frozen=True)
class CachedResult(Generic[ValueT]):
    value: ValueT | None
    source: CacheSource
    cache_key: str


class CacheMetrics:
    def record(
        self,
        namespace: str,
        source: CacheSource,
    ) -> None:
        # Replace with the application's metrics client.
        print(
            {
                "metric": "cache_result",
                "namespace": namespace,
                "source": source.value,
            }
        )

Application logs and traces should identify the cache namespace and serving layer without exposing sensitive cache keys.

Common Mistakes

Production Checklist

• Define the purpose of every cache layer.
• Cache each response at the highest safe layer.
• Keep authoritative state outside all cache layers.
• Use tenant-aware and versioned application cache keys.
• Set strict size limits for local memory caches.
• Use shorter local TTLs than distributed-cache TTLs.
• Stagger expiration across layers and add TTL jitter.
• Coalesce concurrent misses at proxy and application layers.
• Publish invalidation only after database commit.
• Retain finite TTLs even with event-driven invalidation.
• Protect the database during Redis or CDN failures.
• Track which layer served each request.
• Monitor hit ratio, latency, evictions, and fallback traffic per layer.
• Test cold starts and complete cache-layer outages.
• Document acceptable stale duration for each cached value.

Conclusion

Multi-level caching improves performance by removing work at several points in the request path. Browser and CDN caches reduce network and origin traffic, reverse proxies protect application servers, local caches remove repeated network calls, and distributed caches reduce database work.

The architecture remains reliable only when each layer has a narrow responsibility, bounded TTL, safe key design, observable behavior, and a controlled fallback path.

Key Takeaway: Design cache layers from the user toward the database, serve each request at the earliest safe layer, and ensure that losing any cache reduces performance without breaking correctness or overwhelming authoritative storage.

More Articles to Read

• Caching Explained: Improving Performance Without Overloading Databases
• Cache in Software System Design — A Practical Guide
• Scalability for Dummies - Part 3: Cache
• Understanding Caching in Scalable Systems
• Cache-Aside vs Write-Through vs Write-Behind
• CDN vs Reverse Proxy vs Application Cache
• Cache Invalidation Strategies for Production Systems
• Preventing Cache Stampedes and Hot Keys
• Caching Best Practices for Distributed Applications