DNS, Load Balancers, and Reverse Proxies

Production traffic rarely travels directly from a client to an application server. A request typically passes through several network layers responsible for finding the service, selecting healthy capacity, terminating connections, and routing the request to the correct backend.

DNS, load balancers, and reverse proxies solve different parts of this problem. DNS maps stable service names to reachable endpoints. Load balancers distribute connections or requests across healthy targets. Reverse proxies sit in front of backend services and provide application-aware routing, TLS termination, connection management, and other traffic controls.

Modern infrastructure often combines these responsibilities into the same managed service, but understanding them separately makes it easier to reason about scalability, latency, failure recovery, and production bottlenecks.

Table of Contents

• The Request Routing Path
• DNS
  • DNS Resolution
  • DNS Caching and TTL
  • DNS Traffic Steering
• Load Balancers
  • Layer 4 vs Layer 7
  • Load-Balancing Algorithms
  • Health Checks
• Reverse Proxies
  • Request Routing
  • Connection Management
• DNS vs Load Balancer vs Reverse Proxy
• Failure and Recovery
• Production Design Example
  • Architecture
  • Request Flow
  • Failure Flow
• Common Mistakes
• Production Checklist
• Conclusion
• More Articles to Read

The Request Routing Path

DNS, load balancers, and reverse proxies operate at different stages of the request lifecycle.

Client
  |
  | 1. Resolve api.example.com
  v
DNS
  |
  | 2. Return reachable endpoint
  v
Load Balancer
  |
  | 3. Select healthy capacity
  v
Reverse Proxy
  |
  | 4. Route application request
  v
Backend Service

DNS makes a destination-discovery decision. It tells the client where a service can be reached.

A load balancer makes a traffic-distribution decision. It decides which healthy backend should receive a connection or request.

A reverse proxy makes an application-routing decision. It accepts the client request and forwards it to an upstream service, potentially transforming or inspecting the request along the way.

These layers do not always exist as separate infrastructure. A Layer 7 load balancer is also functioning as a reverse proxy, while an edge platform may combine DNS, TLS, caching, routing, and load balancing.

DNS

The Domain Name System (DNS) provides a layer of indirection between service names and network locations.

Applications can depend on a stable hostname such as api.example.com while the servers, load balancers, regions, or IP addresses behind that hostname change over time.

DNS Resolution

A client normally does not contact an authoritative DNS server directly for every lookup. Resolution passes through a resolver and several DNS layers when the answer is not already cached.

Application
    |
    v
Local Resolver
    |
    v
Recursive Resolver
    |
    +---- Root DNS
    |
    +---- TLD DNS
    |
    +---- Authoritative DNS
    |
    v
Address / Endpoint

For example:

api.example.com
      |
      v
203.0.113.25

The client can then establish a TCP or QUIC connection to the returned endpoint.

Common DNS records used in production systems include:

DNS Caching and TTL

DNS is heavily cached. This reduces latency and DNS infrastructure load, but it also means DNS changes do not become visible to every client immediately.

A DNS response includes a Time to Live (TTL):

api.example.com -> 203.0.113.25
TTL             -> 300 seconds

A resolver may reuse that answer until the TTL expires rather than querying authoritative DNS again.

This has important consequences for failover.

12:00:00  DNS returns Region A
12:01:00  Region A fails
12:01:10  DNS changes to Region B

Client still has cached Region A address
          |
          v
Requests may continue toward Region A
          |
          v
Cache expires
          |
          v
New DNS lookup returns Region B

A lower TTL can reduce the expected convergence time, but it does not turn DNS into instantaneous failover. Client applications, operating systems, recursive resolvers, and connection reuse all affect how quickly traffic moves.

DNS is therefore best treated as coarse-grained routing rather than per-request load balancing.

DNS Traffic Steering

DNS can return different endpoints according to routing policies.

Common strategies include:

• weighted routing — distribute resolutions between endpoints according to configured weights
• latency-based routing — prefer an endpoint expected to provide lower latency
• geographic routing — select infrastructure according to client geography
• failover routing — use a secondary endpoint when the primary becomes unhealthy

A global architecture might use DNS to choose a region:

                 api.example.com
                        |
                        v
                   Global DNS
                  /          \
                 /            \
                v              v
           US Region       EU Region
              |               |
              v               v
        Load Balancer    Load Balancer

DNS performs the large-scale routing decision. Regional load balancers then make faster, more granular decisions about individual backend targets.

Advantages:

• stable service names independent of infrastructure
• routing across regions or providers
• simple global traffic distribution
• no mandatory global proxy in the request path

Disadvantages:

• cached answers slow traffic changes
• routing decisions are relatively coarse
• health information may be less granular than application-level checks
• existing connections are unaffected by DNS changes

When to Use: DNS routing is useful for service discovery, regional traffic steering, disaster recovery, infrastructure migration, and exposing stable application endpoints.

Load Balancers

A load balancer provides a stable frontend while distributing traffic across multiple backend targets.

                   Load Balancer
                  /      |      \
                 v       v       v
              API A    API B    API C

This provides two important capabilities: scaling and failure isolation.

When traffic increases, additional targets can be registered. When a target becomes unhealthy, it can be removed without requiring clients to discover a different server.

Layer 4 vs Layer 7

Load balancers can make routing decisions at different layers.

Layer 4 load balancing operates primarily using transport-level information such as IP addresses, ports, TCP connections, and UDP flows.

Layer 7 load balancing understands application protocols such as HTTP and can inspect request information before selecting an upstream.

Layer 7 routing enables one endpoint to expose multiple services:

api.example.com/users/*     -> User Service
api.example.com/orders/*    -> Order Service
api.example.com/tracking/*  -> Tracking Service

Because Layer 7 infrastructure terminates and understands HTTP, it can also perform redirects, header manipulation, authentication integration, compression, or protocol conversion.

Load-Balancing Algorithms

Once multiple healthy targets exist, the load balancer needs a strategy for selecting one.

Round robin distributes traffic sequentially:

Request 1 -> API A
Request 2 -> API B
Request 3 -> API C
Request 4 -> API A

This works well when targets have similar capacity and requests have similar cost.

Least connections prefers a target with fewer active connections. It can perform better when connections are long-lived or request durations vary significantly.

Weighted balancing assigns different traffic shares:

                Load Balancer
                 /          \
                /            \
             90%             10%
              |               |
              v               v
          Version A       Version B

This is useful for heterogeneous server capacity and controlled deployments such as canary releases.

Hash-based routing selects a target using a stable input such as a client identifier or source address. It can provide affinity, although architectures should avoid depending on affinity when stateless processing is practical.

No algorithm guarantees even backend utilization. One request may consume milliseconds while another performs several seconds of CPU or I/O work. Production balancing should therefore be evaluated using actual target saturation, latency, and concurrency.

Health Checks

Load balancing improves availability only when unhealthy targets can be detected and removed.

A health-check system typically evaluates targets repeatedly:

Target
  |
  | health probe
  v
200 OK
  |
  v
Keep in rotation


Target
  |
  | repeated failed probes
  v
503 / timeout
  |
  v
Remove from rotation

A readiness endpoint should answer whether the application can safely accept new traffic.

from fastapi import FastAPI
from fastapi.responses import JSONResponse

app = FastAPI()

@app.get("/ready")
def ready():
    if application_is_draining():
        return JSONResponse(
            {"status": "not_ready"},
            status_code=503,
        )

    if not database_pool_has_capacity():
        return JSONResponse(
            {"status": "not_ready"},
            status_code=503,
        )

    return {"status": "ready"}

Health checks should not automatically fail because every optional dependency is unavailable. Otherwise one shared dependency outage can cause every application target to remove itself from service.

Failure thresholds also matter. Aggressive checks detect failures quickly but can remove healthy instances during short network interruptions. Conservative checks reduce false positives but continue routing traffic toward failed targets longer.

Reverse Proxies

A reverse proxy accepts client traffic on behalf of one or more backend applications.

The client sees the proxy as the server:

Client
  |
  v
Reverse Proxy
  |
  +----------+----------+
  |          |          |
  v          v          v
User API   Order API  Search API

The backend applications can remain private and do not need to expose their individual addresses directly to clients.

Request Routing

A reverse proxy can inspect HTTP information and determine which upstream service should process the request.

For example:

server {
    listen 443 ssl;
    server_name api.example.com;

    location /users/ {
        proxy_pass http://users_service;
    }

    location /orders/ {
        proxy_pass http://orders_service;
    }

    location /tracking/ {
        proxy_pass http://tracking_service;
    }
}

The proxy can also preserve metadata about the original request:

proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

This is important because the backend connection originates from the proxy rather than directly from the client.

Forwarding headers create a trust boundary. Applications should accept client IP and protocol information from forwarding headers only when the request came through trusted proxy infrastructure.

Connection Management

A reverse proxy creates a boundary between client-side and backend-side connections.

Client
   |
   | HTTP/3 + TLS
   v
Reverse Proxy
   |
   | HTTP/2
   v
Backend Service

The protocol used between the client and proxy does not have to match the protocol used between the proxy and application.

For example, a public edge can accept HTTP/3 while forwarding requests to backend services over HTTP/2 or HTTP/1.1.

For deeper protocol behavior, see HTTP/1.1 vs HTTP/2 vs HTTP/3.

The proxy can also reuse persistent upstream connections:

Client A ---\
Client B ----> Reverse Proxy === persistent connection ===> API
Client C ---/

Connection reuse avoids repeatedly paying connection-establishment and TLS costs between infrastructure components.

However, connection pools are finite resources. Large proxy fleets can overwhelm backend connection limits even when request throughput appears manageable.

Suppose 50 proxy instances each allow 200 connections to one backend:

50 proxies x 200 connections = 10,000 possible upstream connections

If the backend can safely handle only 2,000 concurrent connections, the architecture is over-provisioned at the proxy layer.

Connection capacity must therefore be calculated across the entire fleet, not per proxy instance.

DNS vs Load Balancer vs Reverse Proxy

The technologies overlap, especially at Layer 7, but their conceptual responsibilities remain different.

A useful mental model is:

DNS
"Where should traffic enter?"

Load Balancer
"Which healthy capacity should receive it?"

Reverse Proxy
"How should this application request reach the backend?"

A single infrastructure product can answer more than one of these questions.

Failure and Recovery

DNS, load balancers, and proxies operate at different failure scopes and therefore provide different recovery mechanisms.

Consider a multi-zone application:

                     DNS
                      |
                      v
                Load Balancer
                /           \
               v             v
            Zone A        Zone B
           Proxy A       Proxy B
           /    \         /    \
          v      v       v      v
        API 1  API 2   API 3  API 4

If API 1 fails, the load-balancing layer can remove that target while the endpoint remains unchanged.

If Proxy A fails, traffic can move to Proxy B.

If Zone A fails, the regional load-balancing layer can continue using healthy targets in Zone B.

If the entire region fails, recovery may move to a global traffic layer or DNS-based regional failover.

Backend Failure
      |
      v
Target-level failover
      |
      v
Zone Failure
      |
      v
Regional load-balancer failover
      |
      v
Region Failure
      |
      v
Global routing / DNS failover

Recovery generally becomes slower and more operationally complex as the failure domain grows.

DNS failover is especially important to model carefully because cached answers and existing connections can continue targeting the failed region.

For deeper availability patterns, see Designing Highly Available Network Architectures.

Planned removals require another mechanism: connection draining.

Mark target draining
        |
        v
Stop new requests
        |
        v
Finish in-flight requests
        |
        v
Close remaining connections
        |
        v
Terminate target

Without draining, deployments and autoscaling events can create connection resets even when the application itself is healthy.

Production Design Example

Consider a logistics platform serving browser applications, mobile clients, partner integrations, and internal services across North America and Europe.

The public API uses a stable hostname:

api.example.com

The infrastructure must route clients to an appropriate region, survive individual instance and zone failures, and direct different API paths to independently scalable backend services.

Architecture

                           Clients
                              |
                              v
                         Global DNS
                        /          \
                       /            \
                      v              v
                 US Region        EU Region
                      |              |
                      v              v
                Load Balancer   Load Balancer
                  /     \          /     \
                 v       v        v       v
              Proxy A Proxy B  Proxy C Proxy D
                 |       |        |       |
                 +---+---+        +---+---+
                     |                |
             +-------+-------+        |
             |       |       |        |
             v       v       v        v
          Shipment Tracking Carrier Regional
            API      API      API     APIs

Global DNS provides coarse regional traffic steering.

Regional load balancers distribute traffic across healthy capacity and availability zones.

Reverse proxies perform application-aware routing and maintain upstream connections to backend services.

Backend services scale independently according to their workloads.

Request Flow

Suppose a mobile application requests:

GET https://api.example.com/shipments/SH123/tracking

The first decision is DNS resolution:

api.example.com
      |
      v
Global DNS
      |
      v
US regional endpoint

The client establishes a connection to the returned regional endpoint.

The regional load balancer selects healthy capacity:

Request
   |
   v
Load Balancer
   |
   +---- Proxy A
   |
   +---- Proxy B  <- selected

The reverse proxy examines the path and routes the request:

/shipments/SH123/tracking
             |
             v
       Tracking Service

The proxy reuses an available upstream connection where possible, waits for the backend response, and returns it over the client-facing connection.

The routing path therefore contains several independent decisions:

Hostname
   |
   v
Region
   |
   v
Proxy / Target
   |
   v
Backend Service
   |
   v
Backend Instance

Each decision should be observable independently.

Failure Flow

Suppose one Tracking Service instance stops responding.

Tracking B becomes unhealthy
          |
          v
Health checks fail
          |
          v
Tracking B removed
          |
          v
New requests -> Tracking A / C

No DNS update is necessary because the failure is local to the backend pool.

Now suppose an entire availability zone fails. The regional load-balancing layer removes targets in that zone and sends traffic to surviving zones.

The surviving infrastructure must have sufficient recovery headroom. If every zone operates near maximum capacity during normal traffic, zone failover can overload the remaining targets.

Finally, suppose the US region becomes unreachable.

US Region
   X
   |
Global health detection
   |
   v
Regional endpoint marked unhealthy
   |
   v
Global routing changes
   |
   v
New resolutions -> EU Region

Some clients may continue trying the US endpoint because they have cached DNS records or existing persistent connections. Regional recovery therefore cannot assume immediate global traffic convergence.

Production monitoring should include:

• DNS resolution latency and failures
• DNS answers by endpoint or region
• traffic distribution by region and availability zone
• healthy and unhealthy targets
• health-check transition rate
• requests and connections per target
• active proxy connections
• queued proxy requests
• upstream connection-pool utilization
• connection establishment latency
• TLS handshake latency
• proxy-generated 4xx and 5xx responses
• upstream 4xx and 5xx responses
• upstream connection resets
• upstream timeout rate
• connection-draining duration

Proxy metrics should distinguish errors generated by the proxy from errors returned by upstream applications. Otherwise an HTTP 502 spike may identify the symptom without revealing whether the root cause is connection exhaustion, DNS resolution, an unhealthy upstream, or a network timeout.

Common Mistakes

Traffic-management failures often happen because routing components are treated as interchangeable or their caching, health-check, and connection behavior is ignored.

Production Checklist

• Expose stable DNS names. Keep clients independent from individual server addresses.
• Choose DNS TTLs deliberately. Balance caching efficiency with routing-change requirements.
• Design for cached DNS responses. Do not assume endpoint changes propagate immediately.
• Use DNS for coarse traffic steering. Prefer load balancers for rapid backend-level routing.
• Choose the correct load-balancing layer. Use Layer 4 when transport information is sufficient and Layer 7 when application-aware routing is required.
• Distribute targets across failure domains. Avoid concentrating healthy capacity in one zone.
• Configure meaningful readiness checks. Remove targets that cannot safely process new requests.
• Tune health-check thresholds. Balance detection speed against unnecessary target flapping.
• Maintain failover headroom. Surviving targets must absorb traffic after failures.
• Use connection draining. Allow in-flight requests to complete during deployments and scale-in.
• Reuse upstream connections. Avoid unnecessary connection-establishment overhead.
• Budget connections globally. Calculate backend connection capacity across the complete proxy fleet.
• Protect forwarding headers. Trust proxy metadata only from controlled infrastructure.
• Preserve tracing context. Propagate correlation and distributed-tracing identifiers through every hop.
• Monitor traffic distribution. Detect hot targets, uneven zones, and unexpected regional routing.
• Separate proxy and upstream errors. Identify where failures originate.
• Monitor connection saturation. Track active, idle, queued, failed, and reset connections.
• Test target failure. Verify unhealthy instances leave rotation automatically.
• Test zone failure. Confirm surviving infrastructure can handle redistributed traffic.
• Measure regional failover. Validate actual DNS convergence and recovery time where multi-region availability is required.

Conclusion

DNS, load balancers, and reverse proxies solve different parts of production traffic routing. DNS maps stable service names to network endpoints and can steer clients between regions. Load balancers distribute traffic across healthy capacity. Reverse proxies understand application requests and control how those requests reach backend services.

The boundaries often overlap in modern infrastructure, especially when Layer 7 load balancers also provide reverse-proxy capabilities. The important architectural question is not which product owns each label, but where each routing decision happens, what information it uses, and how it behaves during failure.

Key Takeaway: use DNS to determine where traffic enters the system, load balancing to distribute traffic across healthy capacity, and reverse-proxy behavior to control how application requests reach backend services.

More Articles to Read

• Computer Networking Explained for Backend Engineers
• TCP vs UDP: Choosing the Right Protocol
• HTTP/1.1 vs HTTP/2 vs HTTP/3
• Designing Highly Available Network Architectures
• TLS, HTTPS, and Secure Communication
• Networking Best Practices for Production Systems