What Is a Distributed Lock?

A distributed lock is a coordination mechanism used to ensure that only one process, server, container, or worker can perform a particular operation on a shared resource at a time.

Unlike an in-memory mutex, a distributed lock must coordinate processes running on different machines. This makes locking much harder because networks fail, processes crash, clocks differ, and a lock holder can disappear without releasing its lock.

Table of Contents

• Why Distributed Locks Exist
• How a Distributed Lock Works
  • Lock Acquisition
  • Critical Section
  • Lock Release
• Distributed Lock vs Local Lock
• Atomic Lock Acquisition
• Lock Ownership
• Lock Expiration and Leases
• The Expired Lock Problem
• Fencing Tokens
• Lock Renewal
• Distributed Locks with Redis
• Database-Backed Distributed Locks
• Coordination Systems
• Distributed Locks vs Idempotency
• When Not to Use a Distributed Lock
• Production Design Example
• Common Distributed Lock Mistakes
• Conclusion
  • More Articles to Read

Why Distributed Locks Exist

Consider an application running on three servers. Every server periodically checks whether an invoice needs to be generated.

Server A ─┐
Server B ─┼→ Invoice 8472
Server C ─┘

Without coordination, two servers can observe the same state at nearly the same time:

Server A → invoice not generated
Server B → invoice not generated

Server A → generate invoice
Server B → generate invoice

A local mutex cannot solve this problem because each server has its own memory.

Server A
mutex = unlocked

Server B
mutex = unlocked

Both processes can successfully acquire their own local locks.

A distributed lock moves coordination into a shared system:

Server A ─┐
Server B ─┼→ Shared Lock Service
Server C ─┘
               ↓
        invoice:8472

If Server A owns invoice:8472, Server B and Server C cannot simultaneously acquire the same lock.

This pattern appears in scheduled jobs, background workers, resource provisioning, inventory operations, financial workflows, leader election, and other processes where concurrent execution must be controlled.

How a Distributed Lock Works

A basic distributed lock has three operations:

Acquire Lock
     ↓
Perform Work
     ↓
Release Lock

The difficult part is making these operations correct when multiple clients compete for the lock and failures occur between any two steps.

Lock Acquisition

Suppose two workers attempt to process the same order.

Worker A ─┐
          ├→ lock:order:8472
Worker B ─┘

The lock service must perform acquisition atomically.

Conceptually:

IF lock does not exist:
    create lock
    return success
ELSE:
    return failure

If Worker A wins:

Worker A → acquired
Worker B → rejected

There must never be an intermediate state where both clients conclude that they acquired the same exclusive lock.

Critical Section

After acquiring the lock, the worker performs the operation that requires exclusive access.

Acquire order:8472
       ↓
Update external inventory
       ↓
Update order state
       ↓
Release order:8472

The protected work is called the critical section.

Keeping the critical section small is important. Long-running work increases lock contention and creates more opportunities for expiration, network failures, and process crashes.

Lock Release

After completing the work, the owner releases the lock so another client can acquire it.

However, release cannot simply mean:

DELETE lock

The process attempting the deletion must still own that lock.

This ownership requirement becomes critical once locks can expire.

Distributed Lock vs Local Lock

A normal mutex coordinates threads or processes that share a synchronization mechanism on one machine.

Process
  ├→ Thread A
  └→ Thread B
       ↓
      Mutex

A distributed lock coordinates independent processes across machines:

Server A ─┐
Server B ─┼→ Distributed Lock
Server C ─┘

The difference is not just where the lock is stored. Distributed locks operate in an environment where clients can lose communication without actually stopping.

Atomic Lock Acquisition

Lock acquisition must be implemented as one atomic operation.

This approach is unsafe:

if not lock_exists(key):
    create_lock(key)

Two workers can execute the check simultaneously:

Worker A → lock does not exist
Worker B → lock does not exist

Worker A → create lock
Worker B → create lock

The classic check-then-act race appears because checking and creating are separate operations.

The lock store instead needs a primitive equivalent to:

Create this key only if it does not already exist.

The entire decision must happen atomically inside the coordination system.

This principle is similar to other concurrency-control mechanisms: correctness depends on making the ownership decision indivisible.

Lock Ownership

A distributed lock should identify its owner.

Instead of storing:

lock:order:8472 = locked

store a unique ownership token:

lock:order:8472 = 8f96d5e2-...

The token is generated when the client attempts to acquire the lock.

Only a client holding the matching token should be allowed to release it.

Conceptually:

IF lock.value == my_token:
    delete lock

The comparison and deletion must themselves be atomic.

This prevents an old client from deleting a lock currently owned by another client.

Lock Expiration and Leases

A distributed process can crash after acquiring a lock:

Worker A → Acquire Lock
Worker A → Start Work
Worker A → Crash

Lock remains?

If the lock never expires, every other worker can remain blocked forever.

Distributed locks therefore commonly use a lease: ownership lasts for a limited period.

Lock acquired at 12:00:00
Lease = 30 seconds
Expires at 12:00:30

If the owner crashes, the lock eventually disappears or becomes available again.

The lease duration introduces a trade-off.

A very long lease delays recovery after a crashed worker. A very short lease risks expiring while a healthy worker is still performing legitimate work.

Expiration solves abandoned locks, but it creates another important correctness problem.

The Expired Lock Problem

Suppose Worker A acquires a lock with a 30-second lease.

12:00:00 → Worker A acquires lock
12:00:05 → Worker A begins operation

The process then pauses for longer than expected because of CPU starvation, garbage collection, a slow dependency, or another runtime problem.

12:00:30 → Lock expires
12:00:31 → Worker B acquires lock
12:00:32 → Worker A resumes

Now both workers may believe they can continue:

Worker A → still executing old operation
Worker B → legitimate current lock owner

The lock service itself has behaved correctly. Worker A's lease expired.

The problem is that Worker A may not immediately know that it no longer owns the lock.

A lease therefore limits how long the lock service recognizes ownership, but it cannot physically stop an old process from continuing to execute.

This distinction is one of the most important aspects of distributed locking.

Fencing Tokens

Fencing tokens protect downstream resources from operations performed by stale lock holders.

Every successful lock acquisition receives a monotonically increasing number:

Worker A → token 41
Worker B → token 42
Worker C → token 43

Suppose Worker A pauses until its lease expires. Worker B then acquires the lock.

Worker A → token 41 → pauses

Lock expires

Worker B → token 42 → writes successfully

Worker A → resumes → attempts write with token 41

The protected resource remembers the highest accepted token:

last_token = 42

incoming token 41
41 < 42

Reject operation

Worker A can continue running, but its stale operation is prevented from modifying the resource.

Fencing therefore strengthens the safety model beyond simple lease expiration.

The downstream system must participate in the protocol for fencing to work. A fencing token provides no protection if the resource receiving the operation ignores it.

Lock Renewal

Some operations legitimately take longer than the initial lease.

The owner can periodically renew the lock:

Acquire lease: 30s
      ↓
Work
      ↓
Renew at 10s
      ↓
Work
      ↓
Renew at 20s
      ↓
Complete
      ↓
Release

This mechanism is sometimes called a heartbeat or lease extension.

Renewal should only succeed if the caller still owns the same lock.

If renewal fails because the lease has already expired, the worker should assume it no longer has exclusive ownership.

Applications must still consider whether the operation can safely stop at that point. If an external side effect has already started, simply abandoning the function may not undo it.

Distributed Locks with Redis

Redis is frequently used to implement distributed locks because it provides atomic conditional key creation with expiration.

A simplified acquisition command is conceptually equivalent to:

SET lock:order:8472 <unique-token> NX PX 30000

The important properties are:

• NX: create the key only if it does not exist;
• PX: attach an expiration time;
• unique token: identify the lock owner.

Release must compare the ownership token and delete the key atomically.

A simplified conceptual operation is:

IF GET lock:order:8472 == my_token:
    DELETE lock:order:8472

Executing the comparison and deletion as separate client commands would introduce another race, so implementations commonly perform them atomically inside Redis.

Redis-based locking can be practical for many workloads, but the required safety level matters. A lock protecting occasional duplicate background work has different correctness requirements from a lock controlling irreversible financial or infrastructure operations.

The broader reliability trade-offs around partial failures are covered in Handling Partial Failures in Production Systems.

Database-Backed Distributed Locks

An existing relational database can also coordinate distributed workers.

One approach uses a table with a uniqueness constraint:

distributed_locks

resource_key     owner_token       expires_at
------------------------------------------------
order:8472       8f96d5e2...      12:00:30

The database's transaction and uniqueness guarantees determine which worker acquires the resource.

Another approach uses database-specific locking primitives when all competing processes already depend on the same database.

Database-backed locking has an important operational advantage: it may avoid adding another infrastructure dependency solely for coordination.

However, locks also create database traffic and contention. Large numbers of high-frequency locks can turn the database into a coordination bottleneck.

Database transactions and local row-level locking solve related but different problems. See Database Locks and Transactions for the database-level model.

Coordination Systems

Some distributed systems use dedicated coordination technologies rather than general-purpose caches or application databases.

These systems can provide primitives for:

• leases;
• leader election;
• membership;
• ordered updates;
• distributed configuration;
• consensus-backed state.

The underlying requirement is often broader than locking. For example, a cluster may need to select exactly one active coordinator and replace it when it fails.

This is closely related to Leader Election and Distributed Coordination.

A dedicated coordination system can provide stronger semantics, but it also introduces operational complexity. The choice should depend on the required correctness guarantees rather than treating every shared operation as a reason to deploy a separate coordination cluster.

Distributed Locks vs Idempotency

Distributed locks and idempotency solve different problems.

A lock tries to prevent concurrent execution:

Worker A → allowed
Worker B → blocked

Idempotency allows repeated execution while preventing repeated logical effects:

Request 1 → process payment
Request 2 → same idempotency key
          → return existing result

When duplicate requests are the main problem, idempotency can often provide a simpler and more failure-tolerant design than locking.

For example, preventing a payment from being charged twice is usually better modeled around an idempotency key and durable uniqueness guarantee than around a short-lived distributed lock alone.

Idempotency and Deduplication in Distributed Systems explains these techniques in more detail.

Locks and idempotency can also be combined. A lock can reduce concurrent work while idempotency protects correctness when retries, lease expiration, or partial failures still produce duplicate execution.

When Not to Use a Distributed Lock

A distributed lock adds coordination to a system, and coordination usually reduces availability and increases operational complexity.

Before introducing one, consider whether the invariant can be enforced closer to the data.

For example, suppose only one reservation can exist for a particular request.

Instead of:

Acquire distributed lock
        ↓
Check reservation
        ↓
Create reservation
        ↓
Release lock

a database uniqueness constraint may enforce the invariant directly:

CREATE UNIQUE INDEX unique_reservation
ON reservations(request_id);

Concurrent inserts then compete through the database's transactional guarantees.

Other alternatives include:

• idempotency keys;
• unique database constraints;
• optimistic concurrency control;
• compare-and-swap operations;
• queue partitioning by resource key;
• single-writer ownership;
• state-machine transitions with conditional updates.

The best synchronization mechanism is often the one closest to the invariant being protected.

Production Design Example

Consider a platform that generates one daily report per customer. Multiple worker instances consume report-generation jobs.

Queue
  ↓
┌──────────┬──────────┬──────────┐
Worker A   Worker B   Worker C

Retries or duplicate messages can cause two workers to receive work for the same customer and date.

The lock key is:

report:{customer_id}:{date}

For customer 912 on September 24:

report:912:2026-09-24

Worker A attempts to acquire the lock with a unique ownership token and a 60-second lease.

lock_key = f"report:{customer_id}:{report_date}"
owner_token = generate_unique_token()

acquired = acquire_lock(
    key=lock_key,
    owner=owner_token,
    ttl_seconds=60,
)

if not acquired:
    return

Worker A acquires the lock. Worker B receives a duplicate message and fails to acquire it.

Worker A → lock acquired
Worker B → lock unavailable

Worker A generates the report and stores it.

However, the design does not rely on the lock as the only correctness mechanism.

The database also contains a unique constraint:

CREATE UNIQUE INDEX unique_daily_report
ON reports(customer_id, report_date);

This protects the final invariant even if Worker A pauses, its lease expires, and another worker eventually processes the same job.

The resulting protection has two layers:

Distributed Lock
      ↓
Reduce concurrent duplicate work
      ↓
Database Unique Constraint
      ↓
Protect final data invariant

If report generation can exceed 60 seconds, the worker renews its lease periodically while it remains the owner.

Metrics record:

• lock acquisition attempts;
• lock contention rate;
• acquisition latency;
• lease expirations;
• renewal failures;
• critical-section duration;
• duplicate database insert attempts.

A sudden increase in contention can reveal duplicate job production or insufficient partitioning. Frequent lease expiration can indicate that the TTL no longer matches real processing time.

If the workload already uses a message queue, partitioning work by resource key may sometimes remove the need for a separate lock. Message Queues Explained: Producers, Consumers, and Brokers covers the underlying queue model.

Common Distributed Lock Mistakes

• Using check-then-create for acquisition. Lock acquisition must be atomic.
• Creating locks without expiration. A crashed owner can block the resource indefinitely.
• Deleting a lock without checking ownership. An old worker can accidentally release another worker's lock.
• Assuming expiration stops the old process. A stale worker can continue executing after losing its lease.
• Choosing an arbitrarily short TTL. Legitimate work may continue after the lock expires.
• Choosing an extremely long TTL. Recovery from crashed workers becomes unnecessarily slow.
• Renewing a lock without verifying ownership. A stale process must not extend a newer owner's lease.
• Using locks as the only protection for irreversible operations. Idempotency, uniqueness constraints, or fencing may still be necessary.
• Holding locks during unnecessary network calls. Large critical sections increase contention and failure exposure.
• Ignoring the lock service itself as a distributed system. Failover and network partitions affect the guarantees the lock can provide.
• Adding a distributed lock when the database can enforce the invariant directly. Extra coordination may add complexity without improving correctness.

A distributed lock should have a clearly defined safety property: what resource is protected, who owns the lock, how ownership expires, what happens when the owner pauses or crashes, and what prevents stale owners from causing damage.

Conclusion

A distributed lock coordinates processes running across multiple machines so that only one lock owner performs a protected operation at a time.

Implementing one correctly requires more than storing a shared key. Acquisition must be atomic, ownership must be identifiable, abandoned locks need expiration, releases must verify ownership, and applications must handle the possibility that a worker continues running after its lease expires.

For high-value operations, distributed locking is often only one layer of protection. Fencing tokens, idempotency, database constraints, conditional writes, or single-writer designs can provide stronger guarantees closer to the resource being protected.

The core principle is: a distributed lock grants temporary ownership, but correctness depends on what the system does when that ownership is delayed, lost, expired, or observed differently during a failure.

More Articles to Read

• Distributed Systems Explained: Core Concepts and Trade-Offs
• Timeouts, Retries, and Exponential Backoff
• Failure Recovery in Distributed Systems