What is a load balancer and how does it work?

Every server has a limit: CPU, memory and simultaneous connections. When traffic goes past that limit, responses slow down and, in the worst case, the service goes down. You have two ways out: replace the server with a bigger one (vertical scaling) or run several servers that share the work (horizontal scaling). The load balancer is the piece that makes the second option possible.

What is a load balancer?#

A load balancer is a component that receives client requests and forwards them to a group of servers that provide the same service. To the people using it, there is one address, such as api.example.com. Behind it sit several servers, and the client does not know which one answered.

It can be software (nginx, HAProxy, Envoy), a cloud provider service or a dedicated appliance. The idea is the same in all three: one entry point, many destinations.

How does a load balancer work?#

Each request goes through four steps:

1. The client resolves the domain name through DNS and connects to the balancer's address.
2. The balancer checks its list of healthy servers.
3. It picks one using the configured algorithm.
4. It forwards the request, receives the response and returns it to the client.

Step 2 is what provides resilience. The balancer watches server health and removes unresponsive ones from rotation. There are two ways to check: active, where the balancer runs periodic probes (for example, a request to /health), and passive, where it watches real responses and marks a server as failed when it returns errors. Open source nginx does only passive checks (active ones are an NGINX Plus feature): after failures within a time window (max_fails and fail_timeout), it avoids that server for a while.

Which load balancing algorithms exist?#

The algorithm decides which server gets the next request. The three most common:

There are also weighted variants, which send more traffic to more powerful servers. IP hash is one form of sticky sessions, but it has a limit: clients behind the same NAT share an IP and all land on the same server.

What is the difference between layer 4 and layer 7?#

Layer 4 balancing works at the transport level (TCP/UDP). It looks only at IP and port, without understanding the content. That makes it fast and lets it handle any protocol, from databases to game servers.

Layer 7 balancing works at the application level (HTTP). It reads headers, paths and cookies, and can decide based on them: send /api to one group of servers and /static to another, for example. In return, it does more work per request. This kind also usually terminates TLS, which makes it close to a reverse proxy.

How do you set up a load balancer with nginx?#

The upstream block lists the servers, and proxy_pass points to the group. This example balances by fewest connections, with passive health checks:

http {
    upstream app {
        least_conn;
        server 10.0.0.11:3000 max_fails=3 fail_timeout=30s;
        server 10.0.0.12:3000 max_fails=3 fail_timeout=30s;
        server 10.0.0.13:3000 backup;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://app;
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
    }
}

Without the least_conn; line, nginx uses round-robin. The server marked backup only gets traffic when the others are unavailable. Replace the IPs with your own servers and reload with nginx -s reload.

What problems does load balancing bring?#

Splitting traffic across servers requires them to be interchangeable. The main concerns:

• In-memory state. If server A keeps the user's session in its own memory and the next request lands on B, the user loses the authenticated session. The fix is to keep sessions outside the process, in a database or Redis, or to use sticky sessions.
• Local files. An upload written to server A's disk does not exist on B. Use shared storage, such as an object store.
• Scheduled jobs. A cron that runs on every server runs N times. Run it on only one.
• The balancer can fail too. If there is a single instance of it, it is a single point of failure.

When do you not need a load balancer?#

If one server handles the traffic with room to spare and a short outage is acceptable, a balancer only adds complexity and cost. Most bots and small sites are in that situation. Before balancing, measure CPU, memory and latency and check whether adding RAM to the server solves it. A CDN in front of static content also takes a lot of load off the origin server without needing a second one.

How to do this on Square Cloud#

On Square Cloud, load balancing is a feature of the domain, with no nginx to configure and no balancer to maintain. Deploy the same code to two or more applications, set a custom domain on the first one (creating the DNS records the dashboard shows at your registrar), then add the same domain to the others. From then on, requests take turns round-robin across them, and an application that goes down leaves the rotation through passive failover, since there is no active health check.

How many applications can share a domain depends on the plan: up to 2 on Standard (starting at R$44.99 per month), up to 5 on Pro and up to 10 on Enterprise, with no per-request charge. Only a custom domain can be balanced, not the squareweb.app subdomain. And you decide how many applications sit behind the domain: that number does not change on its own with traffic.

The concerns from the previous section apply here. There are no sticky sessions and each application has its own disk, so keep sessions in Redis or another database, send uploads to Blob and run scheduled jobs in a single application. If the bottleneck is one process, raising the application's RAM in its settings may be enough. See the steps in load balancing and each plan's limits on pricing.