Services, Ingress, and Networking
Kubernetes pods are dynamic. They are created during scaling, replaced after failures, moved between nodes, and recreated during deployments. Each replacement can receive a different IP address. Production applications therefore cannot safely depend on discovering and calling individual pod addresses.
Kubernetes networking solves this through several layers. Pod networking provides connectivity between workloads, Services provide stable endpoints and load balancing across changing pod sets, and Ingress provides HTTP and HTTPS routing from outside the cluster toward internal Services.
These abstractions hide much of the infrastructure, but they do not remove network engineering concerns. Traffic still crosses proxies, load balancers, nodes, connection tables, DNS resolvers, and application processes. Latency, connection reuse, topology, health checks, failure propagation, and observability remain important production considerations.
Table of Contents
- Kubernetes Networking Model
- Services
- Ingress
- Traffic Flow and Failure Behavior
- Production Network Design
- Production Design Example
- Common Mistakes
- Production Checklist
- Conclusion
- More Articles to Read
Kubernetes Networking Model
Kubernetes separates workload identity from network service identity. Pods receive addresses that allow them to communicate, but applications normally discover other applications through Services rather than individual pod IPs.
This distinction makes pod replacement possible without forcing every caller to rediscover infrastructure manually. A Service remains stable while its backend pod set changes continuously.
Pod-to-Pod Networking
Each pod normally receives its own cluster-network IP address. Containers inside the same pod share the pod's network namespace and can communicate through localhost.
Communication between different pods uses their pod addresses:
Node A Node B
+-------------------+ +-------------------+
| Pod A | | Pod C |
| 10.20.1.14 | | 10.20.2.31 |
+---------+---------+ +---------+---------+
| |
| Cluster Pod Network |
+----------------+------------------+
|
+----------------+------------------+
| |
+---------+---------+ +---------+---------+
| Pod B | | Pod D |
| 10.20.1.22 | | 10.20.2.45 |
+-------------------+ +-------------------+
The exact implementation depends on the cluster's networking stack, but the application-level model is that pods can address other pods through the cluster network.
This simplifies service-to-service communication compared with manually managing node ports and host addresses. It also means network performance depends on the underlying implementation: cross-node traffic can have different latency, bandwidth, encapsulation, and routing costs from traffic between pods on the same node.
Why Pod IPs Are Not Service Endpoints
Pod addresses are not durable application endpoints. Suppose an inventory API has three replicas:
inventory-api
10.20.1.14
10.20.2.31
10.20.3.18
If the second pod fails, its replacement might become:
10.20.2.57
A caller configured with 10.20.2.31 now has a stale destination. Scaling from three replicas to ten creates the same discovery problem.
A Service solves this by exposing a stable logical endpoint while Kubernetes continuously updates the set of eligible backend pods.
This fits Kubernetes' disposable pod model. More about pod replacement and cluster failure boundaries can be found here: Kubernetes Explained: Pods, Nodes, and Clusters.
Services
A Kubernetes Service represents a stable network endpoint for a changing group of pods. It usually selects pods using labels and exposes them through a stable DNS name and virtual address.
This separates callers from replica lifecycle. A client can continue calling the same Service while deployments, failures, and autoscaling replace the underlying pods.
Service Discovery and Load Balancing
Consider a shipment API managed by a Deployment:
apiVersion: v1
kind: Service
metadata:
name: shipment-api
spec:
selector:
app: shipment-api
ports:
- name: http
port: 80
targetPort: 8000
type: ClusterIP
The selector connects the Service to pods containing the matching label:
metadata:
labels:
app: shipment-api
The resulting architecture is conceptually:
shipment-api Service
|
+-------------+-------------+
| | |
v v v
Pod A Pod B Pod C
:8000 :8000 :8000
The Service does not permanently bind itself to these three pods. When Pod B disappears and Pod D becomes ready, the backend endpoint set changes.
Readiness matters here. Pods that should not receive requests must be removed from normal traffic selection. This allows a new replica to initialize before serving requests and allows unhealthy replicas to stop receiving new traffic.
Health-check semantics are covered in greater depth here: Health Checks, Readiness, and Liveness Probes.
Service Types
Kubernetes supports several Service exposure models. They solve different routing problems and should not be treated as interchangeable configuration options.
| Type | Exposure | Typical Use | Production Trade-Off |
|---|---|---|---|
| ClusterIP | Inside cluster | Service-to-service traffic | Simple and private, but not directly externally reachable |
| NodePort | Port on cluster nodes | Infrastructure integration and special cases | Exposes node-level ports and adds operational complexity |
| LoadBalancer | External load balancer | Direct external exposure | Simple but can create one external load balancer per Service |
| ExternalName | DNS alias | Reference an external DNS service | No normal Kubernetes backend load balancing |
ClusterIP is the normal choice for internal microservices. For example, an orders service can call an inventory Service without knowing where inventory pods run.
LoadBalancer is useful when a workload needs direct external exposure. However, exposing twenty HTTP services with twenty independent external load balancers can increase infrastructure cost and configuration complexity.
This is one reason HTTP workloads often use an Ingress layer: many application routes can share a smaller number of external entry points.
Ingress
A Service answers the question, "How can traffic reach this logical workload?" Ingress addresses a different problem: how should external HTTP or HTTPS traffic be routed to multiple Services?
Ingress can route based on hostnames and URL paths, allowing several applications to share an external traffic entry point.
Ingress Routing
Consider three application Services:
- shipment-api
- tracking-api
- admin-api
An Ingress can expose them through hostname-based routing:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: logistics-ingress
spec:
rules:
- host: api.example.com
http:
paths:
- path: /shipments
pathType: Prefix
backend:
service:
name: shipment-api
port:
number: 80
- path: /tracking
pathType: Prefix
backend:
service:
name: tracking-api
port:
number: 80
- host: admin.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: admin-api
port:
number: 80
The external request flow becomes:
Internet
|
v
External Load Balancer
|
v
Ingress Controller
|
+--- api.example.com/shipments ---> shipment-api Service
|
+--- api.example.com/tracking ----> tracking-api Service
|
+--- admin.example.com/ ----------> admin-api Service
|
Application Pods
This centralizes HTTP routing, TLS termination, and other edge concerns instead of exposing every HTTP application independently.
Ingress is specifically oriented around application-layer routing. It should not automatically become the path for every internal service-to-service request because doing so can add unnecessary hops, latency, and shared failure dependencies.
Ingress Controller
An Ingress object is primarily routing configuration. Actual traffic handling requires an implementation that watches the configuration and programs the data plane. That implementation is the Ingress controller.
This distinction is operationally important. Creating an Ingress resource does not by itself create a functioning reverse proxy unless the cluster has an appropriate controller.
Depending on the environment, the controller may configure proxies, cloud load balancers, or other routing infrastructure.
An Ingress controller can become a critical shared component because many applications may depend on it. Production capacity planning should consider:
- requests per second
- concurrent connections
- TLS handshake load
- request and response sizes
- long-lived connections
- connection timeouts
- backend latency
- number of routes
- controller replica count
A saturated ingress layer can increase latency across otherwise healthy applications. Monitoring only backend pods therefore misses an important part of the request path.
Traffic Flow and Failure Behavior
Kubernetes networking should be analyzed as a request path rather than as isolated resources. Every additional layer can affect latency, connection behavior, observability, and failure propagation.
Internal and external traffic often follow different paths and therefore have different bottlenecks.
Internal Request Flow
Suppose the order API needs inventory information. Instead of discovering inventory pods directly, it calls the inventory Service:
Order Pod
|
| inventory-api
v
Cluster DNS
|
v
Inventory Service
|
+--------+--------+
| | |
v v v
Pod A Pod B Pod C
DNS resolves the Service name, while the cluster networking implementation directs traffic toward an eligible backend.
If Pod B becomes unready, it should stop receiving new normal Service traffic. Existing connections are a separate concern: removing an endpoint does not necessarily terminate every established application connection immediately.
This matters for applications using long-lived HTTP keep-alive connections, HTTP/2, gRPC, database-style connection pools, or WebSockets. Connection reuse can cause traffic distribution to differ substantially from a simple "one request equals one random pod" model.
For high-throughput systems, monitor per-pod request distribution rather than assuming Service-level balancing guarantees perfectly even application load.
External Request Flow
An external API request can cross several independently failing components:
- DNS resolves the public hostname.
- An external load balancer accepts the connection.
- Traffic reaches the ingress data plane.
- Ingress selects the appropriate application route.
- The Service identifies eligible backend endpoints.
- A pod receives the request.
- The application calls databases, caches, queues, or other Services.
A latency increase at any layer contributes to end-to-end request latency. For example, healthy application pods cannot compensate for an overloaded ingress proxy or a network path experiencing packet loss.
Observability should therefore distinguish edge latency, ingress latency, service-to-service latency, application processing time, and downstream dependency latency.
Production Network Design
Kubernetes makes network connectivity easier to declare, but production architecture still needs explicit decisions about traffic paths, connection behavior, isolation, and failure domains.
The simplest route is often the most reliable one. Internal requests should normally avoid unnecessary external routing layers, while public traffic should enter through controlled and observable edge infrastructure.
Connection Behavior and Latency
Network architecture can add small amounts of latency at several layers. At moderate traffic this may be irrelevant, but high-throughput or latency-sensitive systems can expose problems caused by excessive proxying or poor connection management.
Consider an internal request routed unnecessarily through a public ingress:
Service A
|
v
External/Ingress Layer
|
v
Service B
Compared with direct internal Service communication:
Service A
|
v
Service B ClusterIP
|
v
Service B Pod
The first approach can introduce extra network hops, TLS processing, proxy queues, shared capacity constraints, and additional failure points.
Internal traffic should normally use internal Service discovery unless an explicit gateway capability is required.
Connection pools also need sensible limits. Thousands of pods each maintaining large connection pools can create enormous aggregate connection counts against downstream services even when per-pod configuration appears reasonable.
For example:
100 API pods
x 50 connections per pod
--------------------------
5,000 possible downstream connections
Horizontal scaling therefore changes network and dependency capacity requirements, not only application compute capacity.
Security and Network Isolation
Network reachability should not automatically imply application authorization. Authentication and authorization remain application or platform responsibilities even when traffic originates inside the cluster.
At the network layer, Kubernetes NetworkPolicies can restrict which workloads may communicate when supported by the cluster networking implementation.
A simplified policy allowing only order-api pods to reach inventory-api might look like:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: inventory-api-ingress
spec:
podSelector:
matchLabels:
app: inventory-api
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
app: order-api
ports:
- protocol: TCP
port: 8000
This reduces unintended east-west connectivity, but network policy should complement rather than replace API authorization.
Security boundaries and application-level access control are covered more deeply here: Designing Secure API Architectures.
Production Design Example
Consider a logistics platform with shipment, tracking, pricing, and administrative APIs. Public traffic reaches shipment and tracking endpoints, while pricing is used only internally and administration uses a separate hostname.
The network design should minimize unnecessary exposure while maintaining stable discovery as application replicas scale and move across nodes.
Architecture
Internet
|
v
External Load Balancer
|
v
Ingress Controllers
/ \
/ \
api.example.com admin.example.com
/ \ |
v v v
Shipment Tracking Admin
Service Service Service
| | |
API Pods API Pods Admin Pods
|
| internal call
v
Pricing Service
|
Pricing Pods
|
v
Database / Cache / Queue
The pricing Service uses ClusterIP because no external client needs direct access. Shipment and tracking traffic passes through Ingress because they share public HTTP infrastructure. Administration can use different routing and security controls while still sharing appropriate platform components.
Each Service selects pods through stable labels. Deployments can replace those pods without changing application-facing Service names. More about Deployment behavior can be found here: Deployments, ReplicaSets, and StatefulSets.
Scaling, Failures, and Observability
Suppose shipment traffic increases from 2,000 to 8,000 requests per second. The Shipment API scales from 8 to 24 pods.
The application layer is not the only component affected:
- Ingress must handle four times the request volume.
- Service endpoint sets become larger.
- DNS and connection behavior must remain stable.
- Database connection counts may increase.
- Cache traffic may increase.
- Cross-node network throughput may increase.
- Logging and telemetry volume may increase.
If each Shipment API pod can open 30 database connections, scaling from 8 to 24 replicas changes the theoretical connection count from 240 to 720. An application scaling event can therefore overload a database even though Kubernetes networking itself remains healthy.
Now consider an ingress-controller failure. If multiple applications share that ingress layer, a single capacity or configuration problem can affect many otherwise independent Services. The ingress data plane should therefore have sufficient replicas, resource capacity, health checks, and failure-domain distribution.
A pod failure is narrower. When one Shipment API pod becomes unready, the Service removes it from normal backend selection and traffic continues through remaining replicas. If capacity is sufficient, clients may observe little or no impact.
A node failure can remove several backend pods simultaneously. Workload placement therefore influences networking availability because Service abstraction cannot compensate for every backend disappearing at once.
Production monitoring should cover:
- Ingress request rate by host, route, and status code.
- Ingress latency separately from backend processing latency.
- Active and rejected connections at traffic-entry components.
- Service endpoint counts and unexpected loss of ready backends.
- Per-pod traffic distribution to detect hot replicas.
- DNS latency and failures for internal service discovery.
- Cross-node network throughput and packet loss.
- Connection-pool utilization in application workloads.
- Downstream connection counts as replicas scale.
- TLS errors and certificate expiration on HTTPS entry points.
Kubernetes networking availability is an end-to-end property. Healthy Services and pods are insufficient if DNS, ingress, load balancing, node networking, or downstream connections are saturated.
Common Mistakes
Networking incidents often result from treating Kubernetes abstractions as guarantees about application behavior. Services provide stable discovery, but production reliability still depends on correct endpoints, traffic paths, capacity, connection management, and security.
| Mistake | Production Impact | Better Approach |
|---|---|---|
| Calling pod IPs directly | Clients retain stale destinations after pod replacement or scaling. | Use Services for stable workload discovery. |
| Exposing every internal Service externally | Attack surface, cost, and routing complexity increase. | Keep internal workloads on ClusterIP unless external access is required. |
| Routing internal traffic through public Ingress | Extra latency and shared failure dependencies are introduced. | Use internal Service discovery for normal east-west traffic. |
| Assuming Ingress resources process traffic themselves | Routes exist as configuration but no data plane handles requests. | Operate and monitor an appropriate ingress implementation. |
| Ignoring readiness when routing traffic | Starting or unhealthy pods receive production requests. | Define readiness around actual traffic-serving capability. |
| Assuming load is evenly distributed per request | Long-lived connections can create hot replicas. | Monitor per-pod traffic and understand connection reuse. |
| Ignoring aggregate connection pools | Horizontal scaling overwhelms databases and downstream APIs. | Capacity-plan connections as replica counts change. |
| Under-sizing ingress capacity | Shared edge saturation increases latency across many applications. | Scale ingress from measured request, connection, and TLS load. |
| Treating cluster networking as an authorization boundary | Compromised workloads can access services they should not use. | Combine network isolation with application authentication and authorization. |
| Monitoring only application latency | DNS, proxy, load-balancer, and network bottlenecks remain hidden. | Measure latency across each important network layer. |
Production Checklist
A production Kubernetes network should provide stable discovery, controlled exposure, predictable traffic paths, sufficient connection capacity, and visibility into every important routing layer.
- Use Services for discovery. Avoid persistent dependencies on individual pod addresses.
- Default internal workloads to ClusterIP. Expose only services that require external connectivity.
- Keep internal routes internal. Avoid unnecessary ingress or external load-balancer hops for service-to-service calls.
- Validate Service selectors. Confirm that each Service selects exactly the intended workload.
- Monitor endpoint counts. Alert when ready backend capacity drops unexpectedly.
- Design readiness carefully. Remove pods from new traffic before they become incapable of serving requests.
- Capacity-plan ingress. Measure requests, concurrent connections, TLS load, bandwidth, and backend latency.
- Measure per-pod traffic. Detect uneven distribution caused by connection reuse or topology.
- Budget downstream connections. Calculate aggregate pool sizes at maximum expected replica counts.
- Monitor DNS behavior. Track lookup failures and latency for internal service discovery.
- Observe network errors. Monitor connection resets, timeouts, packet loss, and failed TLS handshakes.
- Distribute critical networking components. Avoid placing all ingress capacity in one node or failure domain.
- Restrict east-west connectivity. Apply NetworkPolicies where workload isolation is required and supported.
- Test network failures. Validate behavior when pods, nodes, ingress replicas, and downstream endpoints disappear.
- Measure end-to-end latency. Separate edge, ingress, application, internal-network, and dependency latency.
Conclusion
Kubernetes networking separates rapidly changing workloads from stable application endpoints. Pods provide workload addresses, Services provide stable discovery and backend selection, and Ingress provides HTTP and HTTPS routing into the cluster.
These abstractions make scaling and pod replacement practical, but they do not eliminate network bottlenecks or failure modes. Connection reuse, ingress capacity, DNS, node networking, replica placement, downstream connection pools, security boundaries, and observability all influence production reliability.
Key Takeaway: design Kubernetes networking around stable Services and the shortest appropriate traffic path. Use Ingress for controlled external HTTP routing, Services for workload discovery, and explicit monitoring and capacity planning for the network layers connecting them.
Comments (0)