Circuit Breaker vs Bulkhead vs Load Shedding

Distributed systems fail in different ways. A downstream service can become unavailable, one dependency can consume all shared resources, or incoming traffic can exceed the capacity of an otherwise healthy application. Treating these situations with the same reliability mechanism usually produces poor results.

Circuit breakers, bulkheads, and load shedding address three different problems. A circuit breaker stops repeatedly calling an unhealthy dependency. A bulkhead prevents one workload or dependency from consuming resources required by others. Load shedding rejects excess work before overload causes system-wide collapse.

Production systems often need all three because they protect different boundaries: circuit breakers protect dependency calls, bulkheads protect resource pools, and load shedding protects capacity.

Table of Contents

• Three Different Failure Problems
• Circuit Breaker
  • Advantages
  • Disadvantages
  • When to Use a Circuit Breaker
  • Circuit Breaker Example
• Bulkhead
  • Advantages
  • Disadvantages
  • When to Use a Bulkhead
  • Bulkhead Example
• Load Shedding
  • Advantages
  • Disadvantages
  • When to Use Load Shedding
  • Load Shedding Example
• Circuit Breaker vs Bulkhead vs Load Shedding
• Production Design Example
  • Protecting a Checkout System
• Common Mistakes
• Production Checklist
• Conclusion
  • Key Takeaway
• More Articles to Read

Three Different Failure Problems

Reliability mechanisms should be selected according to the failure being controlled. Circuit breakers, bulkheads, and load shedding are frequently grouped together because all three can reduce cascading failures, but they operate at different points in the system.

Incoming Traffic
      |
      |  LOAD SHEDDING
      |  "Can more work be accepted?"
      v
Application
      |
      +---- BULKHEAD
      |     "How much capacity can this workload consume?"
      |
      v
Dependency Call
      |
      |  CIRCUIT BREAKER
      |  "Should this unhealthy dependency be called?"
      v
Downstream Service

A circuit breaker does not protect an application from receiving too much traffic. A bulkhead does not determine whether a dependency is healthy. Load shedding does not isolate one dependency from another.

The patterns become significantly more useful when these responsibilities remain separate.

Timeouts and retries usually operate alongside these mechanisms. More about bounding remote calls and preventing retry amplification can be found in Timeouts, Retries, and Exponential Backoff.

Circuit Breaker

A circuit breaker observes calls to a dependency and temporarily prevents new calls when failures indicate that the dependency is unhealthy. Instead of allowing every request to wait for the same timeout or produce the same failure, the caller begins failing fast.

A typical circuit breaker has three states:

                  failures exceed threshold
             +-------------------------------+
             |                               |
             v                               |
          CLOSED --------------------------> OPEN
             ^                               |
             |                               |
             | successful probes             | recovery interval
             |                               |
             +--------- HALF-OPEN <----------+
                           |
                           |
                     failed probes
                           |
                           v
                         OPEN

In the closed state, requests flow normally and outcomes are measured. When failures exceed a configured threshold, the circuit becomes open and calls fail immediately. After a recovery interval, a limited number of requests are permitted in the half-open state. Successful probes close the circuit; continued failures reopen it.

Advantages

• Fast failure: requests do not repeatedly wait for a dependency already known to be unhealthy.
• Resource protection: fewer workers, connections, and threads are consumed by doomed calls.
• Reduced downstream pressure: the unhealthy service receives fewer requests while recovering.
• Failure containment: dependency problems are less likely to propagate into callers.
• Explicit recovery probing: half-open behavior provides a controlled path back to normal traffic.

Disadvantages

• Tuning complexity: thresholds that are too sensitive can open during normal error variation.
• Delayed recovery: an overly long open interval can reject requests after the dependency has recovered.
• Distributed state: separate application instances can observe different circuit states.
• False confidence: circuit breaking does not replace timeouts, capacity limits, or graceful degradation.
• Traffic sensitivity: simple failure-count thresholds behave poorly when request rates vary significantly.

When to Use a Circuit Breaker

Circuit breakers are useful for remote dependencies where failures can persist long enough that repeatedly attempting the same operation wastes capacity. External APIs, internal services, databases accessed through remote proxies, and other network dependencies are common candidates.

They are especially valuable when a dependency becomes slow rather than completely unavailable. A failing-fast circuit can prevent hundreds of requests from simultaneously waiting for long timeouts.

Circuit breakers provide less value for failures that are always handled immediately and cheaply or when no useful fallback, rejection, or alternative behavior exists.

Circuit Breaker Example

A simplified circuit breaker can track recent failures and stop calls temporarily:

import time


class CircuitOpen(Exception):
    pass


class CircuitBreaker:
    def __init__(
        self,
        failure_threshold: int = 5,
        recovery_timeout: float = 10.0,
    ):
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.failures = 0
        self.opened_at = None

    def allow_request(self) -> bool:
        if self.opened_at is None:
            return True

        if time.monotonic() - self.opened_at >= self.recovery_timeout:
            # Simplified half-open behavior.
            return True

        return False

    def record_success(self) -> None:
        self.failures = 0
        self.opened_at = None

    def record_failure(self) -> None:
        self.failures += 1

        if self.failures >= self.failure_threshold:
            self.opened_at = time.monotonic()


async def call_dependency(breaker, operation):
    if not breaker.allow_request():
        raise CircuitOpen()

    try:
        result = await operation()
        breaker.record_success()
        return result
    except Exception:
        breaker.record_failure()
        raise

A production implementation normally needs rolling failure windows, concurrency-safe state, limited half-open probes, classification of relevant failures, metrics, and protection against every application instance probing the recovering dependency simultaneously.

Bulkhead

A bulkhead isolates resources so failure or saturation in one workload cannot consume all capacity available to other workloads. The name comes from compartments in ships that prevent flooding in one section from sinking the entire vessel.

In software, the isolated resource can be a connection pool, worker pool, queue, thread pool, concurrency limit, service instance group, or infrastructure partition.

Without bulkheads

             Shared Pool: 100
                    |
          +---------+---------+
          |         |         |
          v         v         v
      Payments   Search   Recommendations

Recommendations becomes slow:
100 connections consumed
Payments cannot proceed


With bulkheads

Payments Pool       Search Pool       Recommendations Pool
     30                 40                    30
      |                  |                     |
      v                  v                     X
 Payments             Search             Recommendations

Recommendation failure stays inside its allocation.

Advantages

• Failure isolation: saturation in one dependency does not automatically exhaust resources for others.
• Protected critical capacity: important workloads can retain dedicated resources during incidents.
• Predictable blast radius: resource exhaustion is limited to the affected partition.
• Independent tuning: concurrency and pool sizes can reflect different dependency characteristics.
• Operational visibility: saturation can be measured separately for each workload.

Disadvantages

• Lower utilization: reserved capacity can remain unused while another pool is saturated.
• Capacity planning complexity: every partition needs appropriate limits.
• More configuration: connection pools, worker pools, and concurrency limits must be managed separately.
• Incorrect partitioning: poorly selected boundaries can protect low-value work while starving critical operations.

When to Use a Bulkhead

Bulkheads are valuable when workloads share a limited resource but have different importance, latency, or failure characteristics.

For example, payment authorization and recommendation generation should not necessarily share the same outbound connection capacity. A recommendation outage should not prevent payment requests from acquiring a connection.

Bulkheads are also useful between interactive and background workloads:

Database Capacity
       |
       +---- API connections -------- protected
       |
       +---- Background workers ----- limited
       |
       +---- Reporting jobs --------- limited

Without isolation, a large reporting query or background batch can consume resources needed by latency-sensitive API requests.

Bulkhead Example

A semaphore can provide a simple concurrency bulkhead for an asynchronous dependency:

import asyncio


class DependencyBulkhead:
    def __init__(self, max_concurrency: int):
        self._semaphore = asyncio.Semaphore(max_concurrency)

    async def execute(self, operation):
        async with self._semaphore:
            return await operation()


payment_bulkhead = DependencyBulkhead(max_concurrency=50)
recommendation_bulkhead = DependencyBulkhead(max_concurrency=20)


async def authorize_payment(operation):
    return await payment_bulkhead.execute(operation)


async def get_recommendations(operation):
    return await recommendation_bulkhead.execute(operation)

This prevents Recommendation operations from consuming Payment's concurrency allocation. Production implementations should also bound how long callers wait to enter the bulkhead. Otherwise an unlimited queue can simply move resource exhaustion from active operations into waiting requests.

Load Shedding

Load shedding intentionally rejects work when accepting it would push the system beyond sustainable capacity. The goal is to preserve useful throughput and predictable latency for work that can still be processed.

Overloaded systems often experience nonlinear degradation. As utilization approaches saturation, queues grow, latency increases, requests time out, and clients retry. Those retries generate additional load and can drive the system into collapse.

Normal

Traffic
  |
  v
[ Service ]
Capacity available
Latency stable


Overload without shedding

Traffic >>> Capacity
       |
       v
    Queue grows
       |
       v
Latency increases
       |
       v
Timeouts increase
       |
       v
Retries increase
       |
       +----------> More traffic


Overload with shedding

Traffic >>> Capacity
       |
       +---- accepted within capacity
       |
       +---- rejected quickly

Advantages

• Protects latency: bounded accepted work prevents queues from growing indefinitely.
• Preserves useful throughput: available resources process requests likely to complete.
• Prevents cascading overload: the system avoids consuming all resources on work that will eventually time out.
• Supports prioritization: critical traffic can receive capacity before optional workloads.
• Provides explicit failure: clients receive immediate rejection instead of unpredictable long waits.

Disadvantages

• Intentional request rejection: some valid requests fail during overload.
• Capacity estimation: limits must reflect actual sustainable throughput.
• Client behavior matters: aggressive retries can turn rejected traffic into another overload wave.
• Priority design: determining which requests should be rejected first requires business context.

When to Use Load Shedding

Load shedding is useful whenever a service has finite capacity and excessive concurrency can make the entire workload fail. Common triggers include request concurrency, queue depth, database connection saturation, worker utilization, memory pressure, or downstream capacity limits.

Shedding can happen at multiple layers:

• edge or API gateway rate limits
• service concurrency limits
• bounded request queues
• background-worker queue limits
• priority-based admission control
• dependency-specific request limits

The best rejection point is usually as early as possible once the system knows the work cannot be completed within acceptable limits.

Load Shedding Example

A service can use a concurrency limit to reject requests instead of allowing unlimited waiting:

import asyncio


class Overloaded(Exception):
    pass


class ConcurrencyLimiter:
    def __init__(self, limit: int):
        self.limit = limit
        self.active = 0
        self._lock = asyncio.Lock()

    async def acquire(self) -> None:
        async with self._lock:
            if self.active >= self.limit:
                raise Overloaded()

            self.active += 1

    async def release(self) -> None:
        async with self._lock:
            self.active -= 1


limiter = ConcurrencyLimiter(limit=500)


async def handle_request(operation):
    await limiter.acquire()

    try:
        return await operation()
    finally:
        await limiter.release()

In a real HTTP service, overload can map to an appropriate temporary failure response. Clients should not immediately retry rejected traffic without a bounded retry strategy because that would undermine the capacity protection.

Circuit Breaker vs Bulkhead vs Load Shedding

The patterns are easier to choose when framed as three questions:

Circuit Breaker:
"Is this dependency healthy enough to call?"

Bulkhead:
"How much of the system can this dependency consume?"

Load Shedding:
"Can the system afford to accept this work at all?"

The patterns should not be treated as alternatives where exactly one must be selected. A production dependency can have a timeout, bounded retries, a circuit breaker, an isolated connection pool, and a concurrency limit simultaneously.

Production Design Example

Consider an e-commerce checkout system that calls Inventory, Payments, Fraud Detection, and Recommendations. Payment and Inventory are required for checkout, while Recommendations are optional. Fraud Detection is required but can occasionally become significantly slower than the other dependencies.

The architecture should prevent failures in any one dependency from consuming the resources needed by the entire checkout service.

Protecting a Checkout System

                         Client
                           |
                           v
                     API Gateway
                           |
                    Rate Limiting
                           |
                           v
                    Load Shedding
                     max concurrency
                           |
                           v
                    Checkout Service
                           |
          +----------------+----------------+
          |                |                |
          v                v                v
      Inventory         Payments          Fraud
      Bulkhead          Bulkhead         Bulkhead
        100               100               40
          |                |                |
     Circuit Breaker  Circuit Breaker  Circuit Breaker
          |                |                |
          v                v                v
      Inventory         Payments          Fraud
       Service           Service          Service

                           |
                           +---- Recommendations
                                      |
                                  Bulkhead: 20
                                      |
                                Circuit Breaker
                                      |
                                      X
                               Recommendation
                                  unavailable
                                      |
                                      v
                              Skip recommendations

The first protection boundary is admission control. Suppose Checkout can sustainably process 500 concurrent operations while meeting its latency target. Once that capacity is exhausted, additional low-priority requests should be rejected instead of joining an unlimited queue.

Within Checkout, dependencies receive separate concurrency allocations. Fraud becoming slow cannot consume the connections reserved for Payment. Recommendations receive a smaller pool because recommendation generation should never threaten checkout capacity.

Each remote dependency also has an independent circuit breaker. If Recommendations begins timing out repeatedly, its circuit opens and future calls fail immediately. Checkout then uses a degraded response without recommendations.

Inventory and Payment failures cannot use the same fallback because their results are required for a correct checkout. Their open circuits cause checkout to fail quickly rather than spending resources on requests that cannot currently complete.

The combined request path becomes:

1. Admission control
      |
      | capacity available?
      v
2. Enter dependency bulkhead
      |
      | capacity available?
      v
3. Check circuit breaker
      |
      | dependency considered healthy?
      v
4. Execute with timeout
      |
      | transient failure?
      v
5. Bounded retry with backoff
      |
      v
6. Success / fallback / fast failure

Retries must remain bounded by the same end-to-end deadline. A circuit breaker should not be used as justification for aggressive retries before the failure threshold is reached.

Suppose Recommendations normally receives 3,000 requests per second. During an outage, a 700 ms timeout without circuit breaking can create approximately 2,100 concurrent waiting calls before considering retries:

3,000 requests/sec x 0.7 sec
= 2,100 concurrent waiting calls

If the Recommendation bulkhead allows only 100 concurrent calls, the blast radius is already limited. Once the circuit opens, most subsequent requests avoid the dependency entirely.

This illustrates why the patterns complement each other. The bulkhead limits damage while failure is being detected; the circuit breaker reduces continued calls after detection; load shedding protects the checkout service if total demand exceeds its own capacity.

When Recommendations is unavailable, checkout can continue with reduced functionality because the dependency is optional. More about designing such fallback behavior can be found in Designing Graceful Degradation Strategies.

Production monitoring should expose each protection mechanism independently:

checkout.admitted_requests
checkout.shed_requests
checkout.active_requests

payment.bulkhead.active
payment.bulkhead.rejected
payment.circuit.state
payment.circuit.open_count

fraud.bulkhead.active
fraud.bulkhead.wait_time
fraud.circuit.state

recommendation.bulkhead.rejected
recommendation.circuit.state
recommendation.fallback_rate

A high fallback rate with stable checkout success indicates graceful degradation. Increasing load-shed requests indicate insufficient available capacity or abnormal demand. Frequent circuit transitions can indicate an unstable dependency or poorly tuned breaker configuration.

Common Mistakes

These reliability patterns become dangerous when they hide overload, create excessive rejection, or move resource exhaustion from one queue into another.

Production Checklist

Circuit breakers, bulkheads, and load shedding should be designed from actual dependency behavior and capacity limits rather than enabled with identical defaults across every service.

• Identify dependency failure modes: distinguish persistent failures, slow responses, resource starvation, and total service overload.
• Set timeouts first: ensure remote calls cannot consume resources indefinitely.
• Define circuit failure criteria: count technical failures that indicate dependency health rather than valid business outcomes.
• Use rolling breaker thresholds: consider failure ratio and request volume instead of isolated errors.
• Limit half-open probes: restore traffic gradually after a dependency begins recovering.
• Choose bulkhead boundaries: isolate workloads with different priorities, latency profiles, or failure characteristics.
• Bound waiting queues: prevent concurrency isolation from creating unlimited queued work.
• Protect critical capacity: reserve sufficient resources for business-critical paths.
• Measure sustainable capacity: determine where throughput stops scaling and latency begins increasing sharply.
• Shed work early: reject requests before they consume scarce downstream resources.
• Coordinate retries: ensure rejected or circuit-broken traffic does not immediately return as retry amplification.
• Define degraded behavior: decide whether each dependency failure causes fallback, partial response, queued processing, or fast failure.
• Monitor protection mechanisms: expose circuit state, bulkhead saturation, queue depth, rejected requests, and fallback rates.
• Alert on business impact: distinguish optional feature degradation from failure of critical operations.
• Load-test failure behavior: verify the system remains stable when dependencies become slow while traffic remains high.

Conclusion

Circuit breakers, bulkheads, and load shedding solve different reliability problems. Circuit breakers stop repeated calls to dependencies that appear unhealthy. Bulkheads constrain how much shared capacity one dependency or workload can consume. Load shedding prevents the system from accepting more work than it can process safely.

The strongest production designs combine these patterns with timeouts, bounded retries, idempotency, graceful degradation, and observability. Their purpose is not to eliminate errors but to make the impact of failures predictable and prevent local problems from expanding across the system.

Key Takeaway

Use circuit breakers to stop calling unhealthy dependencies, bulkheads to isolate resource consumption, and load shedding to reject work beyond sustainable capacity. Apply the patterns at different protection boundaries and combine them so dependency failures, resource saturation, and traffic overload remain contained instead of becoming cascading outages.

More Articles to Read

• Building Reliable Systems: Core Reliability Patterns Explained
• Timeouts, Retries, and Exponential Backoff
• Designing Graceful Degradation Strategies
• Health Checks, Readiness, and Liveness Probes
• Failure Recovery in Distributed Systems
• Reliability Best Practices for Production Systems