How a load balancer shares incoming requestsDistribution, health and application state are separate jobs

Put the chooser in the request path

A client contacts a service address. A load-balancing component selects a backend from an eligible pool and forwards or routes the traffic using its supported mechanism. Layer 4 and Layer 7 designs make different decisions with different visibility. Our diagram uses a simplified HTTP reverse-proxy model; it is not a packet-level description of every load-balancing product.

The backend still performs application work. A balancer cannot invent database capacity or make a slow dependency disappear. If all three backends wait on the same overloaded database, spreading incoming requests may move the queue without removing the underlying limit. Diagnose the full path before buying more capacity in the wrong place.

Follow equal-weight round robin

Our local scheduler has three eligible servers, A, B and C, with equal weights. Six independent requests are assigned A, B, C, A, B, C. This is a deterministic teaching model of rotation. Real implementations can account for weights, connections, retries and multiple worker processes; do not expect an exact global sequence from every production balancer.

Equal request counts are not necessarily equal work. One request may return a small text file while another performs a long query. Policies such as least connections or weighted distribution can suit different workloads, but they also need meaningful configuration. Naming an algorithm does not establish that it is best for a particular service.

Health checks narrow the eligible pool

Suppose B is declared unavailable. Our fixture removes it from the pool; the next four new assignments are A, C, A, C after resetting the demonstration’s rotation. No assignment targets B. If the pool becomes empty, the fixture reports no healthy backend instead of returning a fictional success.

The fixture is given health state directly. It does not discover failures over a real network. A production system needs suitable active or passive health behavior, thresholds and recovery handling. A TCP connection can succeed while the app cannot serve a useful response, and a deep check can fail because of a dependency rather than the server itself.

Decide where session state lives

Imagine a user signs in on A and the next request reaches C. If the sign-in state exists only in A’s memory, C may not recognize the session. A shared session mechanism or an intentionally chosen persistence policy is part of application design. The balancer’s presence does not automatically replicate memory between machines.

Sticky routing can reduce certain state problems, but it also changes distribution and recovery behavior. If the chosen server disappears, the application still needs a plan. Do not use stickiness as an unexplained substitute for durable state or a clear authentication design. Stateless request handling can simplify some paths without eliminating all shared dependencies.

Treat retries as application-sensitive

If a response is lost after a backend completed a write, sending the request again can repeat its side effect. A balancing retry policy must respect method semantics and the application contract. Our model only assigns synthetic read requests; it does not prove safe retry behavior for orders, payments or emails.

Likewise, keeping a long-lived connection on one backend is different from independently balancing every message sent over it. A chat connection does not jump between servers after each sentence merely because the service has a balancer. Establish the unit being distributed before drawing arrows.

Choose a small operational checklist

Identify the backend pool, policy, weights, health criteria, timeouts and state assumptions. Observe errors and latency alongside utilization. Test an unavailable backend and an empty pool in a permitted environment. Plan redundancy for the balancing layer itself where required; one chooser can otherwise become another single point of failure.

Limits

The executed scheduler is not NGINX, a network appliance or a throughput test. It checks routing decisions for stated inputs. The lesson omits TLS configuration, deployment-specific health-check features and production capacity planning. Those require the selected product’s current documentation and representative testing.

References

Sources used to check the explanations and examples are listed below.

1. NGINX: HTTP balancing methods and weights
2. NGINX: proxying to backend groups

Found a mistake? Send a correction with this lesson’s title, URL and disputed detail. No login needed.