infrastructure
What is Load Balancing? - Definition & Meaning
Load balancing distributes incoming traffic across multiple servers so no single system gets overloaded. It is the foundation for high availability.
What is Load Balancing?
Load balancing is the distribution of incoming network traffic across multiple servers to spread the workload evenly and eliminate single points of failure. This significantly improves the availability, reliability, and performance of applications. A load balancer acts as a traffic controller that intercepts requests and forwards them to the most suitable backend server based on health status, available capacity, and configured routing rules.
How does Load Balancing work technically?
Load balancers operate at different OSI layers: Layer 4 (transport) distributes traffic based on IP and TCP/UDP port without inspecting packet content, resulting in very low latency overhead. Layer 7 (application) makes content-aware decisions based on HTTP headers, URL paths, cookies, or even request body, enabling advanced routing scenarios. Common algorithms include round-robin (distributing requests evenly in fixed order), least connections (routing to the server with fewest active connections), weighted round-robin (servers with more capacity receive proportionally more traffic), IP hash (consistent routing based on client IP for session affinity), and least response time (routing to the server with the fastest response). Health checks continuously monitor backend server health via HTTP endpoints, TCP checks, or custom scripts; unhealthy servers are automatically removed from the pool and re-added once they recover. Session persistence (sticky sessions) ensures a user is always routed to the same server, necessary when session state lives locally on the server. NGINX and HAProxy are popular software-based load balancers with extensive configuration options. Cloud providers offer managed solutions such as AWS ALB/NLB, Google Cloud Load Balancer, and Azure Load Balancer with built-in autoscaling and health monitoring. SSL termination at the load balancer reduces cryptographic overhead on backend servers by handling TLS handshakes centrally. Auto-scaling groups coupled with load balancers automatically add servers during traffic increases and remove them when the spike passes, optimizing costs. Global Server Load Balancing (GSLB) distributes traffic across multiple data centers or regions via DNS-based routing, automatically directing users to the nearest cluster for lower latency and higher availability during regional outages. Canary deployments become possible by sending a small percentage of traffic via weighted routing to a new application version, so errors can be detected before all users are affected. PROXY Protocol (v1/v2) preserves the original client IP address through the entire chain of load balancers and reverse proxies, which is essential for accurate geolocation, access logging, and rate limiting on the actual source address. WebSocket-aware load balancers keep long-lived connections intact through connection upgrade detection and prevent idle timeouts from prematurely terminating them.
How does MG Software apply Load Balancing in practice?
MG Software implements load balancing across all client production environments. We use NGINX as a reverse proxy and load balancer for web applications, and cloud-native load balancers with Vercel and AWS. For Kubernetes deployments, we configure Ingress controllers that intelligently distribute traffic across pods. We set up health checks at the application level (not just TCP) so that servers that hang but remain reachable are automatically taken out of rotation. SSL termination happens at the load balancer, after which internal traffic flows over a trusted network. This ensures our client applications remain available during traffic spikes, deployments, and maintenance windows. We implement canary deployments via weighted routing, initially directing five percent of traffic to a new version while monitoring error rates and latency before rolling out fully. Connection draining is configured by default with a 30-second timeout so in-flight requests always complete gracefully, even during rolling updates in Kubernetes, preventing errors visible to end users during deployments.
Why does Load Balancing matter?
Without load balancing, your application depends on a single server, creating a single point of failure. When that server crashes or traffic spikes, every user is affected. Load balancing eliminates this risk by distributing traffic, automating failover, and enabling horizontal scaling. For businesses, this means higher uptime, a better user experience under load, and the ability to add servers without downtime, directly contributing to customer satisfaction and revenue. Financially, load balancing enables auto-scaling: servers are provisioned only when needed and automatically removed during off-peak hours, significantly reducing monthly cloud costs. It also enables development teams to perform zero-downtime deployments, so updates go live without maintenance windows or planned downtime that drives customers away.
Common mistakes with Load Balancing
Health checks only ping port 80 or verify TCP is open while the application itself is deadlocked, so traffic keeps hitting effectively dead nodes. Sticky sessions are configured without expiration and permanently pin users to a single machine that cannot scale. Teams assume a load balancer automatically scales the database, while the data store remains the actual bottleneck. SSL termination is forgotten and every backend performs unnecessary expensive TLS handshakes. Connection draining is not configured, so in-flight requests are abruptly terminated during deployments. Server weights in weighted routing are never adjusted when capacity changes, leaving powerful machines underutilized while weaker ones are overloaded. Load balancer access logs omit upstream latency, making it impossible to identify slow backends from the balancer metrics alone.
What are some examples of Load Balancing?
- A news website using NGINX round-robin load balancing during breaking news to distribute traffic across ten application servers, serving millions of concurrent visitors without noticeable latency while auto-scaling automatically adds extra servers as the load continues to grow.
- A SaaS platform using AWS Application Load Balancer to route API requests to the correct microservice based on URL path and HTTP headers, with automatic health checks removing unhealthy containers from the pool within seconds and only re-adding them after successful recovery checks.
- An e-commerce site using weighted load balancing to gradually shift traffic to newer, more powerful servers during a migration while keeping old servers as a fallback.
- A video streaming service using least-connections load balancing to distribute transcoding jobs across GPU servers, ensuring no single machine is overloaded while others sit idle.
- A banking application combining IP-hash load balancing with WebSocket support so long-lived connections are consistently routed to the same backend for transaction integrity.
Related terms
Further reading
Frequently asked questions
What is the difference between Layer 4 and Layer 7 load balancing?
Layer 4 load balancing works at the transport level, distributing traffic based on IP address and port number without inspecting request content. It is fast and efficient for straightforward traffic distribution with minimal overhead. Layer 7 works at the application level and can make routing decisions based on HTTP headers, URL paths, cookies, or request body. Layer 7 enables advanced scenarios like path-based routing to different microservices and content-aware decisions, but is slightly slower due to the additional packet inspection.
How does load balancing improve availability?
Load balancing improves availability because health checks continuously monitor the status of each backend server at configurable intervals. When a server fails or stops responding healthily to the health check, it is automatically removed from the pool and all traffic is directed to healthy servers. Users experience no disruption. Once the server recovers and reports enough consecutive successful health checks, it is re-added to the pool. This enables zero-downtime deployments, rolling updates, and scheduled maintenance.
Do I need load balancing for a small application?
For small applications with minimal traffic, a dedicated load balancer is not strictly necessary. However, once you want to guarantee high availability, scale your application horizontally, or perform zero-downtime deployments, load balancing becomes essential. Managed platforms like Vercel, Railway, and cloud services provide built-in load balancing, so you benefit from it without extra configuration. Even for small projects, a simple health check endpoint is valuable for detecting downtime automatically and routing traffic away from unhealthy instances.
What are health checks and how do I configure them?
Health checks are periodic probes the load balancer runs to determine whether a backend server is healthy. The simplest variant checks if a TCP port is open, but a better approach is an HTTP endpoint (for example /health) that validates application status, database connectivity, and other dependencies. Configure an interval (for example every 10 seconds), a threshold for consecutive failures before a server is marked unhealthy, and a recovery threshold. An interval of 5 to 10 seconds strikes a good balance between fast failover detection and low overhead on backend servers.
What is the difference between a load balancer and a reverse proxy?
A reverse proxy sits in front of one or more servers and handles requests on behalf of the backend, including SSL termination, caching, and compression. A load balancer specifically distributes traffic across multiple server instances. In practice, tools like NGINX and HAProxy combine both functions: they act as reverse proxies with load balancing capabilities. Managed cloud load balancers focus purely on traffic distribution and health monitoring.
How does load balancing work in Kubernetes?
Kubernetes provides multiple levels of load balancing for different use cases. A Service of type ClusterIP distributes internal traffic across pods within the cluster. A Service of type LoadBalancer automatically provisions a cloud load balancer from your cloud provider. Ingress controllers (such as NGINX Ingress or Traefik) provide Layer 7 load balancing with path-based routing, SSL termination, rate limiting, and header-based routing. The Kubernetes kube-proxy handles the actual traffic distribution via iptables or IPVS rules on each node.
What is connection draining and why is it important?
Connection draining (also called graceful shutdown) ensures that a server being removed from the pool (due to a deployment, scale-down, or health check failure) first completes all in-flight requests before it stops receiving new traffic. Without connection draining, active requests are abruptly terminated, causing errors for end users. Most load balancers support a configurable drain timeout, typically 30 to 60 seconds. In Kubernetes, set terminationGracePeriodSeconds at the pod level to ensure sufficient drain time during rolling deployments and scale-down events.
This concept in your system?
We work with this every day. In a one-hour conversation you know whether it is a portal, an integration or a rebuild.