Consistency in System Design

By Oleksandr Andrushchenko — Published on — Modified on

Consistency in system design describes how fresh and correct data should be when users, services, or replicas read it after a write. Some systems require every read to reflect the latest committed value, while others can tolerate stale data for better latency, availability, or scalability.

Table of Contents

Why Consistency Matters

Consistency affects correctness, user experience, and system complexity. A bank transfer system cannot show inconsistent balances after money moves between accounts. A shopping catalog, on the other hand, can often tolerate slightly stale product descriptions if that improves latency and availability.

Correctness vs User Experience

Some data must be correct immediately. Examples include account balances, inventory reservations, user permissions, and payment status. If these values are stale or conflicting, the system can lose money, violate security rules, or create operational incidents.

Other data is more flexible. A news feed can show posts a few seconds late. A product recommendation can be based on slightly older behavior. An analytics dashboard can lag behind real traffic. In these cases, weaker consistency is often acceptable because users still get a useful experience.

Consistency Is Not One Setting

Large systems rarely use one consistency model everywhere. A checkout flow may use strong consistency for payment and inventory, eventual consistency for email notifications, and weak consistency for analytics events.

Payment status      -> strong consistency
Inventory count     -> strong consistency
Email notification  -> eventual consistency
Analytics dashboard -> weak consistency

The goal is not to make everything strongly consistent. The goal is to apply the right consistency level to each part of the system.

CAP and PACELC

CAP Theorem

The CAP theorem explains that during a network partition, a distributed system must choose between consistency and availability.

This does not mean systems can never be consistent and available at the same time. During normal operation, many systems provide both. CAP becomes important when nodes cannot communicate reliably.

PACELC Theorem

PACELC extends CAP by explaining that systems trade off latency and consistency even when there is no partition.

If a partition occurs, choose between availability and consistency. Else, choose between latency and consistency.

This is why consistency is not only a failure-mode question. Even during normal operation, waiting for more replicas or regions to confirm a write can improve correctness but increase latency.

Common Consistency Models

Strong Consistency

Strong consistency means reads return the latest committed write. This is the easiest model to reason about because the system behaves as if there is one correct current value.

For example, after an account balance changes from 500 to 400, future reads should not show the old value. This is important for banking, billing, inventory, and permission checks.

Linearizability

Linearizability is a strict form of strong consistency. Operations appear to happen in a real-time order. If write A completes before read B starts, read B must observe write A or a later value.

This is useful for leader election, distributed locks, coordination services, and critical metadata where stale reads can break correctness.

Eventual Consistency

Eventual consistency means replicas may temporarily disagree, but if no new writes happen, they eventually converge to the same state.

For example, if a user updates a profile picture, the main database may store the new image immediately while a CDN, search index, or recommendation system still shows the old value for a short time.

Causal Consistency

Causal consistency preserves cause-and-effect relationships. If operation B depends on operation A, clients should not see B before A.

For example, if a user posts a comment and then another user replies to that comment, the reply should not appear before the original comment. This model is weaker than strong consistency but stronger than basic eventual consistency.

Session Guarantees

Session guarantees are practical client-focused consistency rules. They improve user experience without requiring full global strong consistency.

  • Read-your-writes: after a user writes data, that same user sees the update.
  • Monotonic reads: once a user sees a newer value, they should not later see an older value.
  • Monotonic writes: writes from the same client are applied in order.

For many applications, session guarantees provide the right balance between user experience and scalability.

Patterns to Achieve Consistency

Leader-Based Replication

In leader-based replication, one node accepts writes and replicates them to followers. If reads go to the leader, the system can provide stronger consistency. If reads go to followers, the system may return stale data when replication lags.

Writes
  │
  ▼
Leader
  │
  ├── Follower 1
  └── Follower 2

This pattern is common because it is simple and predictable, but the leader can become a bottleneck.

Quorum Reads and Writes

Quorum systems use read and write thresholds across replicas. If there are N replicas, a write quorum is W, and a read quorum is R, then choosing R + W > N helps ensure reads overlap with recent writes.

For example, with three replicas, a system may require two replicas to acknowledge a write and two replicas to answer a read.

N = 3
W = 2
R = 2

R + W = 4
4 > 3

Higher quorum values improve consistency but increase latency and reduce availability during failures.

Consensus Protocols

Consensus protocols such as Raft and Paxos allow multiple nodes to agree on a single sequence of operations. They are used when correctness and ordering matter more than raw availability.

Consensus is common in systems that manage cluster metadata, leadership, configuration, and coordination. The trade-off is additional coordination overhead.

CRDTs

Conflict-free Replicated Data Types, or CRDTs, are data structures that can be updated independently on different replicas and merged deterministically.

They are useful for collaborative and highly available systems where replicas must accept writes independently. Examples include counters, sets, and collaborative editing structures.

Optimistic Concurrency Control

Optimistic concurrency control assumes conflicts are rare. A record is read with a version number, and the update succeeds only if the version has not changed.

def increment_counter(store, key):
    while True:
        value, version = store.get(key)
        new_value = value + 1

        success = store.put_if_version(
            key,
            new_value,
            expected_version=version,
        )

        if success:
            return new_value

This avoids blocking but requires retry logic when another writer updates the same record first.

Practical Techniques

Idempotency

Idempotency makes operations safe to retry. This is important because distributed systems often face uncertain outcomes: a request may time out even though the server completed the operation.

For example, payment APIs often use idempotency keys to prevent duplicate charges when clients retry after a timeout.

Read-Your-Writes

Read-your-writes consistency ensures that after a user updates data, that same user can immediately see the update.

def write_and_read(session, store, key, value):
    node = session.get("preferred_node") or store.leader()

    node.put(key, value)
    session["preferred_node"] = node

    return node.get(key)

This can be implemented by reading from the leader after writes, using session stickiness, or carrying version information in the client session.

Cache Invalidation

Caches introduce consistency challenges because cached data may outlive the source value. If correctness matters, cache invalidation must be designed carefully.

Common approaches include short TTLs, event-based invalidation, write-through updates, and versioned cache keys.

Background Reconciliation

Background reconciliation repairs divergence after the fact. Replicas, caches, indexes, or downstream systems can periodically compare state and correct inconsistencies.

This pattern is common in eventually consistent systems where immediate agreement is too expensive but long-term convergence is required.

Quick Comparison

Model Latency Availability Typical Use Cases
Strong / Linearizable Higher Lower during partitions Banking, inventory decrement, leader election
Sequential Medium to high Medium Systems that need global ordering but not strict real-time ordering
Causal Low to medium High Social feeds, comments, collaborative apps
Eventual Low High Caches, analytics, catalogs, recommendations

When to Use Which Model

Use stronger consistency when incorrect data causes serious business damage. Examples include double spending, incorrect inventory reservations, stale permission checks, duplicate usernames, and payment status.

Use eventual or weaker consistency when temporary staleness is acceptable. Examples include social feeds, recommendations, analytics dashboards, cached product catalogs, notifications, and search indexes.

Data or Operation Recommended Consistency Reason
Account balance Strong Incorrect values can cause financial loss
Inventory reservation Strong Prevents overselling
User profile update Read-your-writes / eventual The user should see their own change quickly, but global propagation can lag
Email notification Eventual Can be sent after the main transaction
Analytics dashboard Weak / eventual Delayed data is usually acceptable

Summary

Consistency is not a single on/off switch. It is a spectrum of models, guarantees, and trade-offs. Strong consistency simplifies reasoning but can increase coordination cost. Eventual consistency improves availability and latency but requires conflict handling and reconciliation. Session guarantees often provide a practical middle ground for user experience.

The best approach is to classify operations by consistency requirements. Keep critical paths strongly consistent, allow non-critical paths to be eventually consistent, and document the guarantees clearly so API consumers know what to expect.

Good system design does not force one consistency model everywhere. It applies the right consistency level to the right part of the system.

For a broader discussion of implementation patterns, see Consistency patterns.

Comments (0)