What Is Redis?

Redis is a high-performance, in-memory data store used for caching, session management, counters, rate limiting, leaderboards, and real-time applications. It provides atomic commands and specialized data structures that allow applications to access and update shared state with very low latency.

Redis is often deployed alongside PostgreSQL or another primary database, but it can also serve as a database itself when its durability and consistency characteristics meet application requirements. Understanding how Redis executes commands, manages memory, persists data, and handles failures is essential before introducing it into a production architecture.

Table of Contents

• How Redis Works
  • Command Execution and Atomicity
  • Memory Management and Expiration
• Redis Data Structures
  • Strings, Hashes, and Sets
  • Sorted Sets and Streams
• Redis Caching Strategies
  • Implementing Cache-Aside
  • Cache Invalidation and Stampedes
• Redis for Counters and Coordination
  • Atomic Rate Limiting
  • Distributed Locks
• Persistence and Durability
  • RDB Snapshots and AOF
  • What Persistence Does Not Guarantee
• Replication and Scaling
  • Replication and Failover
  • Redis Cluster and Hot Keys
• Production Design Example
  • Architecture and Data Flow
  • Handling Failures and Observability
• Ready-to-Use Example
  • Local Setup
  • FastAPI Implementation
  • Testing the Application
• Common Mistakes
  • Non-Atomic Read-Modify-Write
  • Assuming Redis Is Fully Durable
  • Uncontrolled Cache Fallback
  • Using Pub/Sub as a Durable Queue
• Production Checklist
• Frequently Asked Questions
  • Is Redis a Database or a Cache?
  • Is Redis Faster Than PostgreSQL?
  • Can Redis Lose Data?
  • What Is the Difference Between Redis and Memcached?
  • When Should Redis Not Be Used?
• Conclusion
  • More Articles to Read

How Redis Works

Redis keeps its working dataset primarily in RAM. Clients send commands using the Redis Serialization Protocol (RESP), and the server performs operations directly against in-memory data structures.

This architecture avoids disk access for most ordinary commands. Actual request latency still depends on network round trips, serialization, command complexity, server load, and the size of the data being processed.

Command Execution and Atomicity

Redis traditionally uses a single primary command-execution thread per server process. Depending on the version and configuration, additional threads and background processes handle networking, persistence, and supporting operations.

Individual commands execute atomically relative to other commands on the same server. This is particularly useful for counters and data-structure updates shared across multiple application instances.

SET page:481:views 0

INCR page:481:views
INCR page:481:views
INCRBY page:481:views 10

GET page:481:views

The final value is 12. Concurrent requests cannot overwrite one another through the individual INCR operations because Redis applies each increment atomically.

However, executing several commands sequentially does not automatically make the sequence atomic. Redis provides MULTI/EXEC transactions, WATCH-based optimistic concurrency, and server-side scripting when operations must be coordinated.

Memory Management and Expiration

Redis memory usage includes keys, values, metadata, internal data structures, and allocator overhead. Millions of small keys may consume substantially more memory than the sum of their raw values.

Temporary keys can receive a time-to-live (TTL). Redis checks expiration when keys are accessed and also performs background expiration work to remove keys that are no longer needed.

SET session:481 "active" EX 1800

TTL session:481

EXPIRE session:481 3600

DEL session:481

Expiration and eviction are different mechanisms. Expiration removes keys according to their configured lifetime, while eviction removes eligible keys when Redis reaches its configured memory limit.

Eviction policy should reflect the importance of the stored data. Removing a disposable cached product is different from unexpectedly evicting a coordination key or active session.

Redis Data Structures

Redis provides several specialized data structures rather than treating every value as an opaque object. Choosing the correct structure can eliminate application-side sorting, membership checks, and repeated transfers of complete collections.

Command complexity varies by data structure and operation. A single Redis command can still be expensive when it scans a large collection or returns a large response.

Strings, Hashes, and Sets

Strings work well for serialized JSON objects, counters, and tokens. Hashes store multiple fields under one key, allowing individual attributes to be updated without replacing the complete object.

Sets maintain unique members and support membership checks, intersections, and unions. These operations are useful for deduplication, feature eligibility, and relationships that do not require ordering.

HSET user:481 name "Alice" plan "premium"

HGET user:481 plan

HSET user:481 plan "enterprise"

SADD feature:beta:users 481 482 483

SISMEMBER feature:beta:users 481

SREM feature:beta:users 482

A hash is convenient when individual fields change frequently. A serialized JSON value may be simpler when the application reads and validates the entire object as one unit.

Sorted Sets and Streams

Sorted sets associate members with numeric scores and maintain their ordering. They support ranking, score updates, and retrieval of the highest- or lowest-scoring members.

The following Python example maintains a leaderboard and retrieves its ten highest-ranked players without sorting the full collection in application code.

import redis

client = redis.Redis(
    host="localhost",
    port=6379,
    decode_responses=True,
)

leaderboard = "game:2026:ranking"

client.zadd(
    leaderboard,
    {
        "player:481": 1250,
        "player:482": 980,
        "player:483": 1740,
    },
)

client.zincrby(
    leaderboard,
    50,
    "player:481",
)

top_players = client.zrevrange(
    leaderboard,
    0,
    9,
    withscores=True,
)

for player, score in top_players:
    print(player, score)

Streams serve a different purpose. They retain ordered records and support consumer groups, acknowledgments, and pending-entry tracking for asynchronous processing.

XADD orders:events * type order.created order_id 481

XGROUP CREATE orders:events workers 0 MKSTREAM

XREADGROUP GROUP workers consumer-1 COUNT 10 STREAMS orders:events >

XPENDING orders:events workers

Stream consumers still need to handle duplicate delivery, abandoned pending messages, retention limits, and failures between business processing and acknowledgment. Consumer groups do not automatically guarantee exactly-once business execution.

Redis Caching Strategies

Caching is one of the most common reasons to deploy Redis. Applications store frequently requested values in memory to reduce database queries, expensive computations, and repeated external API calls.

The performance benefit depends on the hit rate, regeneration cost, expiration policy, and acceptable staleness. The broader design choices are covered in Cache in Software System Design — A Practical Guide.

Implementing Cache-Aside

Cache-aside keeps the application responsible for populating Redis. On a miss, the application queries its authoritative data source, returns the result, and stores a cached representation for later requests.

The database remains the source of truth. Redis failures can therefore be handled through database fallback, provided the application protects the database from excessive traffic.

import json
import random

from redis.asyncio import Redis
from redis.exceptions import RedisError


async def get_product(
    redis_client: Redis,
    repository,
    product_id: int,
):
    key = f"product:v1:{product_id}"

    try:
        cached = await redis_client.get(key)

        if cached is not None:
            return json.loads(cached)

    except RedisError:
        pass

    product = await repository.get_product(product_id)

    if product is None:
        return None

    ttl = 300 + random.randint(0, 60)

    try:
        await redis_client.set(
            key,
            json.dumps(product),
            ex=ttl,
        )

    except RedisError:
        pass

    return product

The randomized TTL spreads expiration across independently cached entries. It does not prevent multiple concurrent requests from regenerating one particularly popular key.

For a more detailed discussion of cache misses, population, and invalidation, see What Is Cache-Aside?.

Cache Invalidation and Stampedes

Cached data can become stale when the authoritative database changes. A common strategy deletes the relevant Redis key after the database transaction commits, allowing a subsequent read to repopulate it.

However, invalidation alone does not prevent races. A slow request may read an older database version and repopulate the cache after a newer transaction has already invalidated the key.

• Versioned values: Prevent older representations from replacing newer ones.
• Finite TTLs: Bound how long an invalidation failure can leave stale data available.
• Request coalescing: Allow one request to regenerate a popular missing value.
• Stale-while-revalidate: Serve an older value while a background refresh runs.
• Durable invalidation events: Coordinate cache updates across multiple services.

A cache stampede occurs when many requests simultaneously regenerate an expired or missing value. Instead of protecting the database, the cache miss creates a burst of expensive backend work.

TTL jitter, request coalescing, controlled regeneration, and stale-while-revalidate address different parts of this problem. They are often combined in read-heavy systems rather than treated as mutually exclusive alternatives.

Redis for Counters and Coordination

Redis is useful for shared state that must be updated by multiple application instances. Atomic commands reduce the need for application-side synchronization and can support counters, rate limits, and temporary ownership records.

These operations require more care when their correctness depends on several commands or on surviving a primary failure. Atomic execution and durable acknowledgment are separate guarantees.

Atomic Rate Limiting

A fixed-window rate limiter tracks how many requests a user makes during a defined interval. Redis counters are a natural fit because multiple API instances can update the same key atomically.

Incrementing a counter and assigning its expiration should happen together. A Lua script avoids leaving an unexpired counter if the application fails between separate commands.

import time
import redis

client = redis.Redis(
    host="localhost",
    port=6379,
    decode_responses=True,
    socket_connect_timeout=1,
    socket_timeout=1,
)

RATE_LIMIT_SCRIPT = """
local count = redis.call("INCR", KEYS[1])

if count == 1 then
    redis.call("EXPIRE", KEYS[1], ARGV[1])
end

return count
"""

increment = client.register_script(
    RATE_LIMIT_SCRIPT
)


def allow_request(
    user_id: int,
    limit: int = 100,
    window_seconds: int = 60,
) -> bool:
    window = int(time.time()) // window_seconds

    key = f"rate-limit:{user_id}:{window}"

    count = increment(
        keys=[key],
        args=[window_seconds + 1],
    )

    return int(count) <= limit

This implementation allows up to 100 requests per user per fixed 60-second window. Requests around a window boundary can still produce bursts, making sliding-window or token-bucket algorithms preferable for some APIs.

Redis failure behavior must be defined explicitly. Sensitive endpoints may reject requests when rate limiting is unavailable, while lower-risk endpoints may use a temporary local fallback.

Distributed Locks

Redis can coordinate workers through conditional key creation. SET with NX and EX creates a key only when it does not exist and gives the resulting lease an expiration.

Each lock owner should use a unique token. Releasing a lock without checking ownership can accidentally remove a newer lease acquired after the original one expired.

import uuid
import redis

client = redis.Redis(
    host="localhost",
    port=6379,
    decode_responses=True,
)

lock_key = "lock:report:481"
owner = str(uuid.uuid4())

acquired = client.set(
    lock_key,
    owner,
    nx=True,
    ex=30,
)

if acquired:
    try:
        print("Lock acquired")
        # Perform bounded work.
    finally:
        client.eval(
            """
            if redis.call("GET", KEYS[1]) == ARGV[1] then
                return redis.call("DEL", KEYS[1])
            end

            return 0
            """,
            1,
            lock_key,
            owner,
        )

The release script checks ownership atomically. However, a worker may continue executing after its lease expires, allowing another worker to begin the same operation.

Critical operations may therefore require fencing tokens, database constraints, or stronger coordination guarantees. A Redis lease should not be treated as universal protection against concurrent execution.

Persistence and Durability

Redis can persist its dataset to disk, but persistence is optional. A disposable cache may not need disk recovery, while session state or business counters may require more carefully defined durability.

Persistence settings should reflect acceptable data loss, recovery time, storage overhead, and the behavior expected during crashes or failover.

RDB Snapshots and AOF

Redis supports RDB snapshots and Append Only File (AOF) persistence. Both mechanisms can be enabled together, providing different trade-offs between write overhead and recovery behavior.

RDB stores periodic dataset snapshots. AOF records write operations for recovery and can reduce the potential data-loss window when configured with frequent disk synchronization.

RDB is useful for backups and bulk recovery. AOF is often selected when losing changes since the latest snapshot would be unacceptable.

What Persistence Does Not Guarantee

With the commonly used everysec AOF synchronization policy, Redis generally attempts to synchronize data approximately once per second. A crash can still lose recently acknowledged operations.

Replication does not eliminate that risk because replicas normally receive updates asynchronously. A promoted replica may not contain every write acknowledged by the previous primary.

• Process crash: Unsynchronized writes may disappear.
• Storage failure: Local persistence files may become unavailable.
• Primary failover: A promoted replica may lack recent writes.
• Large dataset recovery: Loading persisted data may delay application readiness.

Durability requirements should be expressed through recovery-point and recovery-time objectives. Backups, failover, and restoration need testing rather than relying on persistence configuration alone.

Replication and Scaling

Redis can scale through replication and data partitioning. Replication creates additional copies of data, while sharding distributes different keys across multiple primary nodes.

These techniques solve different bottlenecks. Replicas may serve additional read traffic, while partitioning increases aggregate memory capacity and distributes writes.

Replication and Failover

Redis primary-replica replication is normally asynchronous. Replicas can temporarily return stale values, particularly during network disruption or periods of heavy write traffic.

Redis Sentinel can monitor non-clustered deployments and coordinate automatic failover. Availability improves, but clients must handle topology changes and possible loss of recently acknowledged writes.

• Replica reads: Appropriate when some staleness is acceptable.
• Failover: Applications must reconnect to the new primary.
• Replication lag: Monitor delays affecting freshness and recovery.
• Write acknowledgment: WAIT can request replica acknowledgment without guaranteeing full transactional durability.

Redis Cluster and Hot Keys

Redis Cluster divides the keyspace into 16,384 hash slots. Each primary owns a subset of slots, allowing data and write traffic to be distributed across several nodes.

Certain multi-key operations require all participating keys to belong to the same hash slot. Hash tags can deliberately colocate related keys when those operations are necessary.

user:{481}:profile
user:{481}:settings
user:{481}:sessions

The matching hash tag places these keys in the same slot. Excessive colocation can nevertheless concentrate traffic and memory usage on one primary.

A hot key remains a bottleneck even when the cluster has spare aggregate capacity. Techniques such as local caching, read replication, and carefully managed key duplication can help distribute the pressure.

Production Design Example

Consider a FastAPI product-catalog service backed by PostgreSQL. Multiple application instances serve product descriptions, prices, and availability summaries to a read-heavy application.

Assume an illustrative workload of 6,000 product-read requests per second, with a relatively small number of popular products receiving most requests. PostgreSQL remains authoritative, while Redis stores frequently requested product representations.

Architecture and Data Flow

Each API instance shares the same Redis deployment. Cache keys contain a schema version and product identifier, while tenant-specific or authorization-sensitive representations include additional isolation dimensions.

Cached values receive a five-minute base TTL with randomized jitter. Product updates trigger invalidation after the authoritative database transaction commits.

1. Read: The API checks Redis for a versioned product key.
2. Cache hit: Redis returns the representation without querying PostgreSQL.
3. Cache miss: The API queries PostgreSQL and populates Redis.
4. Update: PostgreSQL commits the authoritative product change.
5. Invalidation: A durable update event removes or refreshes affected keys.
6. Recovery: Missing or expired entries are regenerated from PostgreSQL.

Cache invalidation needs protection against delayed events and concurrent regeneration. Version checks or controlled refresh operations can prevent older representations from overwriting newer ones.

Handling Failures and Observability

A Redis outage can suddenly shift thousands of requests per second to PostgreSQL. Without bounded fallback concurrency, the resulting database pressure may cause a second outage.

The application should use timeouts, concurrency limits, load shedding, and request coalescing. Particularly expensive responses may also use stale-while-revalidate when temporary staleness is acceptable.

Useful metrics include cache hit rate, command latency, memory consumption, evictions, database fallback rate, and invalidation failures. These measurements connect Redis health to application outcomes.

Additional operational guidance is available in Caching Best Practices for Production Systems.

Ready-to-Use Example

The following implementation creates a small FastAPI application backed by Redis. It demonstrates connection pooling, atomic counters, JSON caching, expiration, and explicit Redis failure handling.

The example is self-contained and does not require PostgreSQL. A generated page summary represents data that a production application would normally retrieve from an authoritative repository.

Local Setup

Start Redis locally using Docker and install the Python dependencies. The container binds Redis to the loopback interface rather than exposing it publicly.

docker run -d \
  --name redis-demo \
  -p 127.0.0.1:6379:6379 \
  redis:7

pip install fastapi uvicorn redis

Production deployments should use a maintained Redis version, private networking, authentication, TLS where required, and persistence settings appropriate to the workload.

FastAPI Implementation

The application initializes a shared Redis client during startup and closes it during shutdown. Connection and command timeouts prevent Redis operations from waiting indefinitely.

The counter endpoint requires Redis, while the summary endpoint can regenerate its response when Redis is unavailable.

import json
import logging
import os
from contextlib import asynccontextmanager

from fastapi import FastAPI, HTTPException
from redis.asyncio import Redis
from redis.exceptions import RedisError


logger = logging.getLogger(__name__)

REDIS_URL = os.getenv(
    "REDIS_URL",
    "redis://localhost:6379/0",
)


@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.redis = Redis.from_url(
        REDIS_URL,
        decode_responses=True,
        socket_connect_timeout=1,
        socket_timeout=1,
        health_check_interval=30,
        max_connections=50,
    )

    try:
        yield
    finally:
        await app.state.redis.aclose()


app = FastAPI(lifespan=lifespan)


@app.get("/health/redis")
async def redis_health():
    try:
        await app.state.redis.ping()
        return {"status": "healthy"}

    except RedisError:
        raise HTTPException(
            status_code=503,
            detail="Redis unavailable",
        )


@app.post("/pages/{page_id}/views")
async def record_page_view(page_id: int):
    key = f"demo:page:{page_id}:views"

    try:
        count = await app.state.redis.incr(key)

    except RedisError:
        raise HTTPException(
            status_code=503,
            detail="Counter unavailable",
        )

    return {
        "page_id": page_id,
        "views": count,
    }


@app.get("/pages/{page_id}/summary")
async def get_page_summary(page_id: int):
    key = f"demo:page:{page_id}:summary"

    try:
        cached = await app.state.redis.get(key)

        if cached is not None:
            return {
                "source": "redis",
                "data": json.loads(cached),
            }

    except RedisError:
        logger.warning("Redis cache read failed")

    summary = {
        "page_id": page_id,
        "title": f"Page {page_id}",
        "published": True,
    }

    try:
        await app.state.redis.set(
            key,
            json.dumps(summary),
            ex=120,
        )

    except RedisError:
        logger.warning("Redis cache write failed")

    return {
        "source": "generated",
        "data": summary,
    }

The page-view endpoint uses INCR, so concurrent requests cannot overwrite one another through an application-level read-modify-write race. The counter's durability still depends on Redis persistence and failover behavior.

The summary endpoint treats Redis as an optional cache. A production implementation should query authoritative storage and protect that dependency against excessive fallback traffic.

Testing the Application

Save the application as main.py and start the development server. The first summary request generates the response, while subsequent requests retrieve the cached representation until expiration.

uvicorn main:app --reload

curl http://localhost:8000/health/redis

curl -X POST \
  http://localhost:8000/pages/481/views

curl http://localhost:8000/pages/481/summary

curl http://localhost:8000/pages/481/summary

Restarting Redis may remove cached data, depending on persistence configuration. The summary endpoint can regenerate its response, while the counter may lose previously recorded increments.

Common Mistakes

Most serious Redis problems come from incorrect assumptions about atomicity, durability, and dependency behavior rather than complicated command syntax. Four mistakes are especially important in production systems.

Non-Atomic Read-Modify-Write

Reading a value with GET, modifying it in application code, and writing it back with SET creates a race condition. Two concurrent requests may read the same value and overwrite each other's changes.

Use atomic commands such as INCR when possible. For operations involving multiple commands, consider Lua scripts, MULTI/EXEC, or WATCH-based optimistic concurrency.

Assuming Redis Is Fully Durable

Persistence and replication do not guarantee that every acknowledged write survives every failure. A primary can fail before its latest updates reach a replica or durable storage.

Keep critical business records in storage with suitable durability guarantees unless Redis's persistence and recovery behavior explicitly meets the application's requirements.

Uncontrolled Cache Fallback

When Redis becomes unavailable, an application may redirect every cache miss to PostgreSQL. The sudden increase in database traffic can turn a cache outage into a wider service outage.

Use bounded fallback concurrency, request coalescing, timeouts, and load shedding. Database capacity must be considered when designing cache failure behavior.

Using Pub/Sub as a Durable Queue

Redis Pub/Sub delivers messages to currently connected subscribers without retaining them for later consumption. A disconnected subscriber can permanently miss a message.

Use Redis Streams or another appropriate durable messaging system when acknowledgment, recovery, and consumer coordination are required.

Production Checklist

Redis production readiness depends on more than command performance. The following checks cover the configuration and operational controls that should be reviewed before deployment.

• Memory: Configure maxmemory, eviction policy, and sufficient operational headroom.
• Connections: Reuse connection pools and configure bounded timeouts.
• Persistence: Select RDB and AOF settings based on acceptable data loss.
• Expiration: Define TTLs and retention policies for temporary keys.
• Monitoring: Track memory, evictions, command latency, and cache hit rate.
• Security: Restrict network access and configure authentication and TLS.
• Recovery: Test failover, backup restoration, and client reconnection.

Frequently Asked Questions

The following questions address common Redis misconceptions, comparisons with other technologies, and limitations that affect production adoption.

Is Redis a Database or a Cache?

Redis can function as both. It supports persistent data storage and specialized data structures, but many applications use it primarily as a cache in front of an authoritative database.

Its suitability as a primary database depends on durability requirements, query patterns, consistency expectations, memory costs, and recovery objectives.

Is Redis Faster Than PostgreSQL?

Redis generally provides lower latency for simple in-memory key lookups and atomic operations. PostgreSQL provides relational queries, indexes, transactions, constraints, and stronger durability capabilities for many workloads.

The two systems are often complementary. Redis reduces repeated access to frequently requested data, while PostgreSQL maintains authoritative records and complex relationships.

Can Redis Lose Data?

Yes. Depending on persistence settings and failure conditions, Redis can lose recently acknowledged writes during crashes, storage failures, or primary failover.

RDB snapshots, AOF persistence, replication, and backups reduce particular risks but do not eliminate every data-loss scenario.

What Is the Difference Between Redis and Memcached?

Both provide fast in-memory access. Redis offers richer data structures, persistence options, replication, scripting, and messaging features, while Memcached focuses primarily on distributed object caching.

Memcached can be suitable for straightforward disposable caching. Redis is often preferable when applications need atomic counters, rankings, shared coordination, or more complex operations.

When Should Redis Not Be Used?

Redis may be inappropriate when workloads require complex relational queries, strict transactional durability, large datasets that are expensive to keep in memory, or consistency guarantees it does not provide.

It can also introduce unnecessary complexity when an existing database already meets performance requirements. Redis adds another dependency that must be secured, monitored, scaled, and recovered.

Conclusion

Redis combines low-latency in-memory access with atomic commands and specialized data structures. It is particularly effective for caching, counters, rate limiting, sessions, leaderboards, and selected asynchronous processing workloads.

Its production value depends on selecting appropriate data structures, managing memory, understanding persistence and replication limitations, and protecting dependent systems during failures.

Key Takeaway: Use Redis when shared low-latency access or atomic data-structure operations provide a measurable benefit. Keep authoritative data in storage with suitable durability guarantees unless Redis's recovery and consistency characteristics explicitly satisfy the application's requirements.

More Articles to Read

These related articles explore cache concurrency, traffic concentration, and techniques for keeping caching systems reliable under production load.

• Preventing Cache Stampedes and Hot Keys