What is a Load Balancer?
A load balancer sits between clients and backend servers. Clients send all requests to the load balancer's IP/hostname; the load balancer forwards each request to one of the backend servers and returns the response. From the client's perspective, there is one server. From the backend's perspective, load is spread across the pool.
Benefits: (1) Horizontal scaling — add backend servers to increase capacity. (2) No single point of failure — if one server fails, the load balancer stops routing to it. (3) SSL termination — the load balancer handles TLS, freeing backend servers from crypto overhead. (4) Session management — some load balancers maintain sticky sessions.
graph LR Client --> LB["Load Balancer"] LB --> S1["Server 1"] LB --> S2["Server 2"] LB --> S3["Server 3"] LB --> S4["Server 4"]
Load Balancing Algorithms
Round Robin: distribute requests sequentially (1→2→3→1→2→3...). Simple, good for homogeneous servers with similar request costs. Problem: a long-running request on server 2 doesn't reduce subsequent assignments.
Weighted Round Robin: servers get weights based on capacity. Server with 2× CPU gets 2× share of requests.
Least Connections: route each new request to the server with the fewest active connections. Better than round robin for heterogeneous request processing times (long-polling vs API calls).
Least Response Time: route to the server with the lowest average response time AND fewest connections. Most sophisticated, requires the load balancer to track response times.
IP Hash: hash the client's IP address, route to the same server every time. Provides sticky sessions without session data in the load balancer. Problem: if a server goes down, all its sticky clients are rerouted.
Random: pick a server at random. Statistically equivalent to round robin at scale, zero state.
Health Checks
A load balancer continuously checks the health of each backend server. Types: (1) TCP check — can we open a TCP connection to port 80? (2) HTTP check — does GET /health return HTTP 200? (3) Deep health check — does the health endpoint verify DB connectivity and cache availability?
If a server fails N consecutive checks (e.g., 3 checks, 5 seconds apart), the load balancer removes it from the pool and stops routing traffic to it. When the server recovers and passes checks, it is re-added. This is automatic failure recovery without human intervention.
Layer 4 vs Layer 7 Load Balancing
Layer 4 (Transport): routes based on IP address and TCP port. Very fast (no HTTP parsing), but cannot route based on URL path or HTTP headers. Use case: high-throughput TCP traffic, database connections.
Layer 7 (Application): parses the HTTP request (URL, headers, cookies) and routes accordingly. Can send /api requests to the API server pool and /static to the CDN origin pool. Can do content-based routing: all image requests → image server cluster. Can terminate SSL. More CPU overhead but much more flexible.
In system design interviews, most web services use Layer 7 load balancers (AWS ALB, Nginx, HAProxy in HTTP mode).
Redundant Load Balancers
A single load balancer is itself a single point of failure. Solution: active-passive pair of load balancers. Both connect to all backend servers. Primary handles all traffic. If primary fails (detected via heartbeat), the passive LB takes over within seconds using a floating IP (virtual IP that moves between the pair). DNS still points to the virtual IP, so clients are not affected. Active-active pair: both handle traffic, DNS round-robins between them. If one fails, DNS TTL determines recovery time — short TTL (30s) minimizes impact.