Designing Highly Available Network Architectures
Highly available network architecture is designed so that the failure of a single host, network device, availability zone, gateway, or traffic path does not make an application unreachable.
For backend systems, availability depends on more than running multiple application instances. Traffic must still be able to resolve the service, reach a healthy entry point, cross the correct network path, and connect to dependencies when parts of the infrastructure fail.
The central design principle is remove single points of failure from the traffic path and provide tested alternate paths that can take over automatically.
Table of Contents
- Availability at the Network Layer
- Redundant Traffic Entry Points
- Multi-Zone Network Design
- Redundant Network Paths
- Failure Detection and Traffic Shifting
- Multi-Region Network Availability
- Production Design Example
- Common Mistakes
- Production Checklist
- Conclusion
- More Articles to Read
Availability at the Network Layer
Application availability depends on the complete request path. Running three healthy backend instances provides little value if every request must pass through one failing gateway or one unhealthy load balancer.
A network path should therefore be analyzed as a chain of dependencies:
Client
|
v
DNS
|
v
Edge / CDN
|
v
Load Balancer
|
v
Network Route
|
v
Application
|
v
Database / Cache / External Service
If one mandatory component has no alternative, it becomes a potential availability bottleneck.
Failure Domains
A failure domain is a group of infrastructure that can fail together because it shares a dependency.
Examples include:
- a single virtual machine
- a physical host
- a network switch
- an availability zone
- a data center
- a region
- a DNS provider
- an internet transit path
Two application instances running on different virtual machines but connected through the same single network appliance may still share one failure domain.
High availability therefore requires redundancy across the actual failure boundaries, not merely duplication of application processes.
Single Points of Failure
A single point of failure is any required component whose failure breaks the path for all traffic.
Consider:
Internet
|
v
Single Reverse Proxy
|
+---------+
| |
v v
App 1 App 2
The two application servers are redundant, but the reverse proxy is not.
A more resilient design uses redundant entry points:
Internet
|
v
Redundant Load Balancing Layer
|
+---------+---------+
| | |
v v v
App 1 App 2 App 3
Availability improves only when redundancy exists throughout the critical path.
Redundant Traffic Entry Points
Public applications need a resilient mechanism for converting a hostname into reachable healthy infrastructure.
This usually involves DNS plus one or more redundant load-balancing or edge layers.
Load Balancers and Health Checks
A load balancer distributes incoming traffic across multiple healthy targets.
Clients
|
v
Load Balancer
|
+---------+---------+
| | |
v v v
API A API B API C
If API B fails, the load balancer should remove it from rotation:
Before failure:
A B C
^ ^ ^
healthy
After B fails:
A X C
^ ^
healthy
The load-balancing layer itself must also be redundant. A highly available managed load balancer typically distributes its own components across multiple infrastructure nodes or zones.
Health checks should verify whether a target can actually serve traffic rather than only whether the process exists.
DNS and Failover
DNS can direct clients toward multiple network endpoints or regions.
For example:
api.example.com
|
v
DNS
/ \
v v
Region A Region B
DNS-based failover can remove an unhealthy endpoint and direct future resolutions elsewhere.
However, DNS failover is not instantaneous. Clients, resolvers, operating systems, and applications may cache DNS results.
A low TTL can reduce how long old results remain cached, but it also increases resolver traffic and does not force every client to refresh immediately.
DNS is therefore useful for coarse traffic steering, while load balancers generally provide faster target-level failover inside a region.
The interaction between DNS and load balancers is covered in DNS, Load Balancers, and Reverse Proxies.
Multi-Zone Network Design
A highly available regional architecture should usually avoid placing all critical workloads inside one availability zone.
Zones provide separate infrastructure boundaries so that a localized outage does not necessarily affect the entire region.
Spreading Services Across Zones
A simplified multi-zone architecture looks like:
Load Balancer
/ \
/ \
v v
Zone A Zone B
------ ------
API A1 API B1
API A2 API B2
If Zone A becomes unavailable, traffic can continue through Zone B.
Capacity planning must account for this failure mode. If normal traffic is split 50/50 across two zones, each zone may need enough spare capacity to absorb substantially more traffic when the other fails.
Running exactly enough capacity for steady-state load can create an availability failure even when redundant infrastructure exists.
Cross-Zone Dependencies
Application servers can be multi-zone while still depending on a single-zone service.
Zone A Zone B
------ ------
API A API B
\ /
\ /
\ /
v v
Single Cache in Zone A
If Zone A fails, API B remains alive but may lose access to the cache.
The same risk applies to:
- databases
- message brokers
- service discovery
- NAT gateways
- internal proxies
- configuration systems
Availability must therefore be evaluated dependency by dependency.
Redundant Network Paths
Redundant compute does not help when all traffic shares one mandatory path through a gateway or network appliance.
Critical connectivity should avoid centralized components that cannot fail independently.
Gateways, NAT, and Egress
Private application servers often use an egress gateway or NAT infrastructure when communicating with external services.
A fragile design looks like:
Zone A Apps ----\
\
> Single NAT Gateway
/
Zone B Apps ----/
|
v
Internet
If that gateway fails, both zones lose outbound connectivity.
A more resilient design distributes egress by failure domain:
Zone A Apps ---> NAT / Gateway A ---> Internet
Zone B Apps ---> NAT / Gateway B ---> Internet
This also avoids routing healthy-zone traffic through a failed or degraded zone.
Similar reasoning applies to VPN gateways, firewalls, proxies, and custom routing appliances.
Private Service Connectivity
Internal services should have redundant routes wherever possible.
For example:
Service A
|
+---- Network Path 1 ----+
| |
+---- Network Path 2 ----+
|
v
Service B
In cloud environments, much of this redundancy may be provided by managed virtual networking, but architectural decisions can still introduce single points of failure through custom appliances or single-zone routing.
Custom network infrastructure deserves special scrutiny because it can accidentally reduce the availability provided by the underlying platform.
Failure Detection and Traffic Shifting
Redundancy does not create availability unless the system can detect failures and stop sending traffic toward unhealthy components.
Failure detection must balance speed against false positives and instability.
Health Check Design
A health check should answer a specific question.
A shallow check might verify only that an HTTP server responds:
GET /health
200 OK
But the service could still be unable to process real requests because its database pool is exhausted.
A deeper readiness check can validate critical local conditions:
def readiness():
if not database_pool.can_accept_requests():
return {"status": "not_ready"}, 503
if application_is_draining():
return {"status": "not_ready"}, 503
return {"status": "ready"}, 200
However, health checks should not blindly test every downstream dependency.
If every application instance marks itself unhealthy because one shared dependency fails, the load balancer may remove all instances even though the application could still provide degraded functionality.
Health checks should reflect whether the target itself should receive new traffic.
Failover Speed and Stability
Consider a health check running every 10 seconds and requiring three failures before removing a target:
Failure occurs
|
v
Check 1 fails
|
10 sec
|
Check 2 fails
|
10 sec
|
Check 3 fails
|
v
Target removed
Failover can take tens of seconds.
Reducing the interval and failure threshold speeds detection but increases sensitivity to temporary network glitches.
A rapidly flapping target can repeatedly enter and leave service, creating unstable traffic distribution.
Production systems often use:
- multiple consecutive failures before removal
- multiple successes before restoration
- connection draining
- graceful shutdown
- bounded failover thresholds
The goal is not the fastest theoretical failover. The goal is fast enough recovery without unstable routing decisions.
Multi-Region Network Availability
Multi-zone architectures protect against many infrastructure failures inside one region. They do not protect against a complete regional outage or major regional connectivity failure.
Systems requiring stronger availability may distribute workloads across regions.
Active-Passive and Active-Active
In an active-passive architecture, one region handles normal traffic while another remains ready for failover.
Global DNS
/ \
v v
Region A Region B
ACTIVE STANDBY
Advantages:
- simpler data ownership
- less cross-region coordination
- lower infrastructure cost than full active-active
Disadvantages:
- failover can take longer
- standby capacity must be maintained and tested
- regional recovery may require operational automation
In active-active architecture, multiple regions serve production traffic simultaneously.
Global Traffic Layer
/ \
v v
Region A Region B
ACTIVE ACTIVE
Advantages:
- regional capacity is already serving traffic
- faster traffic redistribution
- lower latency for geographically distributed clients
Disadvantages:
- significantly more complex data architecture
- cross-region consistency challenges
- more difficult failure handling
- higher operational and infrastructure cost
Multi-region design is covered more deeply in Multi-Region Architecture and Disaster Recovery.
State and Data Consistency
Network failover is much easier for stateless application servers than for stateful systems.
Suppose traffic can move instantly from Region A to Region B, but the latest database state exists only in Region A:
Region A
Application
Database: latest state
replication delay
Region B
Application
Database: older state
Network failover may succeed while application correctness fails.
Multi-region availability therefore requires coordination between:
- traffic routing
- database replication
- session management
- cache strategy
- message processing
- write ownership
Highly available networking cannot compensate for a state architecture that cannot tolerate failover.
Production Design Example
Consider a logistics API that must remain reachable during instance failures and a complete availability-zone outage.
The system uses redundant public entry points, applications spread across three zones, zone-local egress infrastructure, and a multi-zone database.
Architecture
Internet
|
v
Global DNS
|
v
Regional Load Balancer
/ | \
/ | \
v v v
Zone A Zone B Zone C
------ ------ ------
API A1 API B1 API C1
API A2 API B2 API C2
| | |
v v v
Egress A Egress B Egress C
| | |
+------------+-------------+
|
v
External APIs
Multi-Zone Database
The load balancer continuously checks application readiness.
Each zone maintains enough capacity so that the remaining zones can absorb traffic after one zone fails.
Outbound carrier API traffic uses zone-local egress paths so that losing one zone does not remove external connectivity for the others.
Failure and Recovery Flow
Suppose Zone B becomes unavailable.
Normal state:
Zone A: 33%
Zone B: 33%
Zone C: 34%
Zone B fails
|
v
Health checks fail
|
v
Zone B targets removed
|
v
Traffic redistributed
Zone A: ~50%
Zone C: ~50%
If Zones A and C were already operating near capacity, the architecture would still fail despite having redundancy.
Recovery capacity is therefore part of availability design.
Now suppose only API B1 fails rather than the entire zone.
Zone B
API B1 -> unhealthy
API B2 -> healthy
Load balancer removes B1
Traffic continues to B2
The system handles both instance-level and zone-level failures using the same principle: detect the failed component, remove its path from routing, and maintain sufficient healthy capacity elsewhere.
Important metrics include:
- healthy targets by zone
- traffic distribution by zone
- load-balancer health-check failures
- DNS resolution failures
- connection failures by destination
- cross-zone traffic volume
- gateway and NAT connection usage
- egress errors
- network latency between zones
- packet loss where observable
- capacity headroom by zone
- regional failover status
Availability testing should deliberately remove infrastructure rather than only waiting for accidental failures.
Useful exercises include:
- terminate application instances
- remove all targets from one zone
- block access to one dependency
- disable one egress path
- introduce network latency
- simulate DNS or endpoint failure
Failure recovery should be tested under realistic production traffic levels because a failover that works under light load can still collapse when surviving infrastructure receives the full workload.
Common Mistakes
Network availability problems often come from architectures that appear redundant at the application layer but still depend on a shared network path, shared zone, or insufficient recovery capacity.
| Mistake | Production Impact | Better Approach |
|---|---|---|
| Running multiple applications behind one proxy | Proxy failure makes every instance unreachable. | Use a redundant load-balancing layer. |
| Placing all targets in one zone | Zone failure causes complete outage. | Distribute services across failure domains. |
| Sharing one egress gateway across zones | Gateway or zone failure removes outbound connectivity. | Use redundant zone-aligned egress paths. |
| Ignoring surviving-zone capacity | Failover overloads healthy infrastructure. | Maintain enough recovery headroom. |
| Using health checks that only test the process | Broken instances continue receiving traffic. | Check readiness to serve real requests. |
| Making health checks depend on every external service | One shared dependency can remove every application instance. | Test local readiness and support graceful degradation. |
| Using aggressive health thresholds | Temporary network issues cause routing flaps. | Balance detection speed with stability. |
| Expecting DNS failover to be immediate | Clients continue using cached endpoints. | Account for TTLs and resolver caching. |
| Designing network failover without state failover | Traffic reaches a region that cannot serve correct data. | Coordinate traffic and data architecture. |
| Never testing zone failure | Hidden dependencies appear only during real outages. | Run controlled failure exercises regularly. |
Production Checklist
Highly available networking requires redundancy, automated failure detection, enough recovery capacity, and regular testing of alternate traffic paths.
- Map the complete traffic path. Identify DNS, edge services, load balancers, routes, gateways, proxies, applications, and dependencies.
- Identify every failure domain. Understand which components can fail together.
- Remove single network entry points. Use redundant load-balancing and edge infrastructure.
- Distribute workloads across zones. Avoid placing all production capacity in one failure domain.
- Distribute critical dependencies. Databases, caches, brokers, and service-discovery systems should match required availability.
- Provide redundant egress paths. Avoid making every zone depend on one gateway.
- Use meaningful readiness checks. Remove targets that cannot safely serve requests.
- Configure stable health thresholds. Avoid both slow detection and excessive flapping.
- Drain connections during planned removal. Allow in-flight requests to complete where possible.
- Maintain recovery headroom. Surviving infrastructure must handle traffic after failure.
- Monitor health by failure domain. Aggregate metrics by instance, zone, and region.
- Measure cross-zone traffic. Understand both latency and failure implications.
- Monitor gateway capacity. Track connections, throughput, errors, and port usage.
- Account for DNS caching. Do not assume endpoint changes are globally instantaneous.
- Coordinate state with traffic failover. Network routing and data recovery must agree.
- Test instance failures. Verify automatic target removal and replacement.
- Test zone failures. Confirm remaining zones can sustain production load.
- Test dependency isolation. Verify degraded behavior when a remote service is unreachable.
- Test regional recovery where required. Validate DNS, routing, data, and application readiness together.
- Observe failover duration. Measure how long users experience errors before healthy paths take over.
Conclusion
Highly available network architecture is built by removing mandatory single paths through infrastructure. Redundant application servers matter only when DNS, load balancers, routes, gateways, dependencies, and recovery capacity are also resilient to failure.
The strongest designs align redundancy with real failure domains, distribute traffic automatically, detect unhealthy components quickly without excessive flapping, and preserve enough spare capacity for surviving infrastructure to absorb failed workloads.
Key Takeaway: network availability comes from multiple independent traffic paths plus reliable failure detection. Every critical request path should continue functioning when one expected component, zone, gateway, or route disappears.
Comments (0)