Computer Networking Explained for Backend Engineers

Backend applications rarely operate in isolation. A typical request may pass through DNS, a load balancer, a reverse proxy, several application services, a cache, a database, and an external API before a response reaches the client.

Understanding networking helps explain production problems that application code alone cannot: connection timeouts, connection-pool exhaustion, DNS failures, intermittent latency, dropped connections, overloaded proxies, port exhaustion, and failures caused by dependencies located several network hops away.

For backend engineers, the most useful networking model is not memorizing every protocol layer. It is understanding how application data moves between processes, how connections are established, where latency appears, and what happens when part of the path fails.

Table of Contents

• How Backend Traffic Moves
  • Network Layers That Matter
  • Packets, Connections, and Requests
• IP Addresses and Routing
  • Private and Public Networks
  • Routing Between Networks
• Ports and Sockets
  • How Connections Identify Processes
  • Connection Lifecycle and Pooling
• DNS and Service Discovery
  • Name Resolution
  • DNS Failures and Caching
• Latency, Throughput, and Bandwidth
  • Network Round Trips
  • Why Chatty Services Are Expensive
• Network Failures in Distributed Systems
  • Timeouts and Partial Failures
  • Connection and Capacity Failures
• Production Design Example
  • Architecture
  • Request and Failure Flow
• Common Mistakes
• Production Checklist
• Conclusion
• More Articles to Read

How Backend Traffic Moves

Backend code usually works with high-level abstractions such as HTTP requests, database connections, or message-broker clients. Underneath those abstractions, the operating system and network stack move data between machines.

A request such as GET /shipments/123 may ultimately become many packets transmitted across several physical and virtual network devices.

Network Layers That Matter

A simplified model useful for backend development is:

Consider an HTTPS request:

HTTP Request
     |
     v
TLS Encryption
     |
     v
TCP / QUIC
     |
     v
IP
     |
     v
Network Interface
     |
     v
Physical / Virtual Network

Each layer solves a different problem. HTTP defines application semantics. TLS protects communication. TCP provides reliable ordered transport. IP determines where packets should travel.

These layers also create different failure modes. An HTTP 503 response is fundamentally different from a TCP connection timeout or failed DNS resolution.

Packets, Connections, and Requests

A common conceptual mistake is treating a request, connection, and packet as the same thing.

A packet is a unit of data transmitted through the network. A connection is communication state between endpoints, such as a TCP connection. An application request is a higher-level operation such as an HTTP request.

One connection can carry many application requests:

TCP Connection
      |
      +---- HTTP Request 1
      |
      +---- HTTP Request 2
      |
      +---- HTTP Request 3
      |
      +---- HTTP Request 4

And one request may require multiple packets.

This distinction becomes important when diagnosing production behavior. A service processing 5,000 HTTP requests per second does not necessarily create 5,000 new TCP connections every second if connections are reused.

IP Addresses and Routing

An IP address identifies a network interface reachable through an IP network. Applications usually work with DNS names, but those names eventually resolve to addresses that networking infrastructure can route.

Routing determines which next network hop should receive a packet so that it eventually reaches its destination.

Private and Public Networks

Production systems commonly separate public entry points from private application infrastructure.

Internet
   |
   v
Public Load Balancer
   |
   v
Private Application Network
   |
   +--------+---------+
   |        |         |
   v        v         v
 API     Worker    Internal API
   |
   v
Private Database Network

Application servers and databases generally do not need direct public exposure simply because public clients eventually use them.

Private addressing reduces unnecessary exposure and allows traffic to enter through controlled infrastructure such as load balancers, gateways, or reverse proxies.

Network boundaries are not a replacement for authentication and authorization, but they reduce the number of directly reachable components.

Routing Between Networks

When the destination is not directly reachable on the local network, traffic is sent toward a router or gateway capable of forwarding it.

A backend request may therefore cross several hops:

Application
    |
    v
Local Network
    |
    v
Router / Gateway
    |
    v
Intermediate Network
    |
    v
Destination Network
    |
    v
Database / Service

Each hop adds some latency and another possible failure point.

This matters especially across regions or distant data centers. Moving a database farther from an application changes the latency of every network round trip, even when neither application code nor database queries change.

Ports and Sockets

An IP address identifies a network endpoint, but one machine can run many networked processes. Ports allow transport protocols to direct traffic to the appropriate service.

A backend host might simultaneously expose an application on port 8080, connect to PostgreSQL on 5432, Redis on 6379, and HTTPS services on 443.

How Connections Identify Processes

A TCP connection can be identified by its protocol and endpoint addresses and ports:

Client
10.0.1.25:52144

        TCP connection

Server
10.0.3.10:5432

The client port is commonly an ephemeral port allocated for the outgoing connection. The server listens on a known port.

Multiple connections can reach the same server port because their endpoint combinations differ:

10.0.1.25:52144  --->  10.0.3.10:5432
10.0.1.25:52145  --->  10.0.3.10:5432
10.0.1.26:48102  --->  10.0.3.10:5432

This becomes operationally important when systems create enormous numbers of short-lived outbound connections. Client ports, connection-tracking infrastructure, proxies, NAT devices, and destination services all have finite capacity.

Connection Lifecycle and Pooling

Creating a network connection has overhead. Depending on the protocol, connection setup can require multiple exchanges before useful application data begins flowing.

Opening a new database connection for every request is therefore inefficient:

Request
   |
   +-- Open DB connection
   +-- Authenticate
   +-- Query
   +-- Close connection

Request
   |
   +-- Open DB connection
   +-- Authenticate
   +-- Query
   +-- Close connection

Connection pooling reuses existing connections:

Application
     |
     v
Connection Pool
 |   |   |   |
 C1  C2  C3  C4
     |
     v
Database

Pooling reduces connection-establishment overhead and protects dependencies from uncontrolled connection creation.

However, pools themselves require capacity planning. If 100 application instances each maintain 50 database connections, the theoretical total becomes:

100 instances × 50 connections = 5,000 connections

A database supporting 1,000 useful concurrent connections cannot safely absorb that configuration simply because each application's local pool size appears reasonable.

DNS and Service Discovery

Applications rarely hard-code dependency IP addresses. DNS provides a level of indirection between a stable name and network addresses that may change.

A backend service might connect to payments.internal.example rather than a specific server address.

Name Resolution

A simplified connection flow looks like this:

Application
    |
    | payments.internal.example
    v
DNS Resolver
    |
    | IP address
    v
Application
    |
    | connection
    v
Payment Service

This allows infrastructure to move or replace service instances while clients continue using a stable name.

DNS is therefore part of the request dependency chain. If name resolution fails, an otherwise healthy application and healthy destination may still be unable to communicate.

DNS Failures and Caching

DNS responses are commonly cached according to their configured lifetime. Caching reduces resolver traffic and lookup latency but creates a trade-off between efficiency and how quickly clients observe address changes.

Consider a service moving from one endpoint to another:

Old:
api.internal -> 10.0.2.10

New:
api.internal -> 10.0.4.20

Clients still holding a cached old address may continue attempting connections to 10.0.2.10 until their cached result is refreshed.

Applications should not assume that changing a DNS record causes every active process to immediately use the new address. Runtime DNS caching, connection reuse, proxies, and existing long-lived connections can all affect transition speed.

DNS infrastructure, load balancing, and proxying will be covered more deeply in DNS, Load Balancers, and Reverse Proxies.

Latency, Throughput, and Bandwidth

Network performance is not one number. Backend engineers should distinguish latency, bandwidth, and throughput.

A network can have high bandwidth and still have high latency. This is common when communicating across large geographic distances.

Network Round Trips

Suppose application-to-database round-trip latency is 4 ms. A request performs ten sequential database operations that each require one network round trip.

10 operations × 4 ms = 40 ms

That is approximately 40 ms of network waiting before accounting for query execution, application processing, scheduling, or contention.

If the database moves to a location with 25 ms round-trip latency:

10 operations × 25 ms = 250 ms

The code is identical, but the request architecture is no longer equivalent.

This is one reason database placement and service boundaries matter so much in distributed systems.

Why Chatty Services Are Expensive

Network latency becomes particularly expensive when operations are sequential.

shipment = get_shipment(shipment_id)
customer = get_customer(shipment.customer_id)
carrier = get_carrier(shipment.carrier_id)
pricing = get_pricing(shipment)
tracking = get_tracking(shipment.tracking_number)

If each remote operation takes 20 ms and all calls are sequential, network waiting alone can approach 100 ms.

Possible improvements depend on data dependencies:

• execute independent requests concurrently
• batch multiple lookups into one operation
• cache frequently reused data
• move strongly coupled operations closer together
• avoid unnecessary service boundaries
• use asynchronous processing where synchronous results are unnecessary

Microservices therefore introduce a network cost that in-process function calls do not have. Service boundaries should reflect architectural ownership and scaling needs rather than maximizing the number of independently deployed services.

Network Failures in Distributed Systems

A local function call usually either returns or throws an error. Network communication has a more difficult property: the caller may not know what happened remotely.

A timeout does not necessarily mean the destination never processed the request. The request may have arrived and completed while the response was lost or delayed.

Timeouts and Partial Failures

Consider a payment request:

Order Service
     |
     | POST /payments
     v
Payment Service
     |
     | charge succeeds
     v
Payment Provider

Response back to Order Service
          X
       timeout

The order service sees a timeout, but the payment may already exist.

Blindly retrying can create a duplicate charge unless the operation supports idempotency.

This illustrates a fundamental distributed-systems rule: a network timeout represents uncertainty, not proof of failure.

Production clients need explicit connection and request timeouts. Without them, slow dependencies can hold application resources for long periods and eventually cause cascading failures.

import httpx

timeout = httpx.Timeout(
    connect=1.0,
    read=3.0,
    write=3.0,
    pool=1.0,
)

with httpx.Client(timeout=timeout) as client:
    response = client.get("https://inventory.internal/items/123")

Timeout values should come from latency expectations and request budgets rather than arbitrary large defaults.

Retries, backoff, and retry safety are covered in Timeouts, Retries, and Exponential Backoff.

Connection and Capacity Failures

Not every networking problem is packet loss. Production applications commonly experience networking symptoms caused by capacity exhaustion.

Examples include:

• connection pools reaching their maximum size
• load balancers reaching connection limits
• proxies running out of workers or file descriptors
• destination services refusing new connections
• NAT infrastructure exhausting available mappings or ports
• network interfaces reaching throughput limits
• connection-tracking tables filling under high connection churn

Connection reuse is therefore both a performance optimization and a capacity-control mechanism.

Long-lived connections introduce different trade-offs. They reduce establishment overhead but can preserve stale routing decisions, consume persistent resources, and require careful timeout and keepalive behavior.

Production Design Example

Consider a logistics platform receiving public shipment requests. The application uses a load-balanced API tier, an internal pricing service, PostgreSQL, Redis, a message broker, and several external carrier APIs.

The request path contains multiple independent network interactions, each with different latency and failure characteristics.

Architecture

                        Client
                           |
                           v
                          DNS
                           |
                           v
                     Load Balancer
                           |
                           v
                      Shipment API
                    /      |       \
                   /       |        \
                  v        v         v
              Pricing    Redis    PostgreSQL
              Service
                  |
                  v
             Message Broker
                  |
                  v
           Tracking Workers
                  |
        +---------+---------+
        |         |         |
        v         v         v
     Carrier A Carrier B Carrier C

Each dependency has a different networking profile.

The application uses connection pools for PostgreSQL and reusable HTTP connections for internal and external HTTP services.

Dependencies are addressed by stable service names rather than instance IP addresses. Public traffic enters through redundant load-balancing infrastructure, while internal traffic remains on private networks.

Request and Failure Flow

A shipment creation request follows this simplified path:

1. The client resolves the public API hostname.
2. The client establishes secure communication with the public endpoint.
3. The load balancer selects a healthy Shipment API instance.
4. The API acquires a database connection from its local pool.
5. PostgreSQL stores the shipment.
6. The API publishes tracking work to the message broker.
7. The API returns the shipment identifier.
8. Workers process carrier communication asynchronously.

Carrier API latency is intentionally removed from the synchronous shipment-creation path. A carrier taking eight seconds to respond should not force a client creating a shipment to wait eight seconds when immediate carrier confirmation is unnecessary.

Now suppose the database becomes slow.

Database latency increases
          |
          v
Queries hold connections longer
          |
          v
Connection pool utilization rises
          |
          v
Requests wait for connections
          |
          v
API latency increases
          |
          v
Client/proxy timeouts begin
          |
          v
Retries may increase traffic

The first visible symptom may be HTTP timeout errors, but the underlying bottleneck can be database latency causing connection-pool saturation.

Useful production metrics therefore span multiple layers:

• DNS resolution latency and failures
• connection establishment latency
• active and new connections
• connection-pool utilization and wait time
• request latency by dependency
• request timeout rate
• connection errors and resets
• retry volume
• network throughput
• packet loss where observable
• load-balancer backend health
• dependency error rates

Network monitoring should be correlated with application behavior. High request latency with low CPU utilization can indicate that application processes are spending most of their time waiting for remote systems rather than computing.

Common Mistakes

Networking problems often appear as application problems because networking libraries hide most transport details. Understanding the underlying connection and request lifecycle makes these failures easier to identify.

Production Checklist

Backend networking should be designed around predictable communication, bounded resource usage, and explicit failure behavior.

• Understand every critical request path. Identify DNS, proxies, services, databases, caches, brokers, and external dependencies involved.
• Keep latency-sensitive components close. Avoid unnecessary geographic network round trips between tightly coupled systems.
• Use stable service discovery. Avoid coupling applications to replaceable instance IP addresses.
• Reuse network connections. Use connection pooling and persistent connections where appropriate.
• Size pools globally. Calculate total possible connections across all application replicas.
• Configure explicit timeouts. Bound connection establishment, pool waiting, reads, and writes where supported.
• Design safe retries. Retry only appropriate failures and protect non-idempotent operations.
• Reduce sequential network calls. Batch or parallelize independent work when it improves latency safely.
• Keep private services private. Expose only infrastructure that requires external connectivity.
• Account for DNS caching. Design endpoint changes around resolver and connection behavior.
• Monitor connection pools. Track utilization, waiting, creation rate, and exhaustion.
• Monitor transport failures. Measure connection errors, resets, timeouts, and failed resolutions.
• Observe dependency latency separately. Averages across the entire request path hide individual bottlenecks.
• Control connection churn. Avoid unnecessary short-lived connections under high traffic.
• Protect downstream capacity. Bound concurrency and connection counts according to dependency limits.
• Test partial failures. Simulate slow, unreachable, and intermittently failing dependencies.
• Design for uncertain outcomes. Use idempotency and reconciliation when requests can complete despite lost responses.
• Correlate network and application metrics. Diagnose latency across layers rather than treating every timeout as an application-code problem.

Conclusion

Networking is part of backend application architecture. Every remote database query, cache lookup, API call, message publication, and service request depends on name resolution, addressing, routing, transport connections, and finite network resources.

Backend engineers do not need to manage every physical network detail, but production systems become much easier to design and debug when connections, ports, DNS, latency, routing, pooling, timeouts, and partial failures are treated as first-class architectural concerns.

Key Takeaway: every remote call is a distributed operation with latency, resource costs, and uncertain failure behavior. Reliable backend systems minimize unnecessary network interactions, reuse connections efficiently, bound waiting time, protect dependency capacity, and observe the complete path between communicating processes.

More Articles to Read

• TCP vs UDP: Choosing the Right Protocol
• HTTP/1.1 vs HTTP/2 vs HTTP/3
• Designing Highly Available Network Architectures
• DNS, Load Balancers, and Reverse Proxies
• TLS, HTTPS, and Secure Communication
• Networking Best Practices for Production Systems