Load balancer timeouts and errors explained

You open your site and see a bare page: 504 Gateway Timeout. Or 502 Bad Gateway. Or it works nine times out of ten and fails on the tenth. Your app logs show nothing, because the request often never reached your code, or reached it too late.

These errors are produced by the thing in front of your app: a load balancer, a reverse proxy or a CDN. Each one has its own clock, and the first one to run out decides the error. The status code and the time until failure tell you which clock it was.

504 Gateway Timeout
502 Bad Gateway
503 Service Unavailable

What do 502, 503 and 504 mean?#

All three are server-side errors defined in RFC 9110, and when a proxy returns them, they usually mean "the proxy is fine, something behind it is not".

A short way to remember it: 502 is "I got garbage or nothing", 503 is "nobody can serve this", 504 is "I waited and you never answered".

Some CDNs use their own codes. Cloudflare returns 524 when it connected to your origin but got no HTTP response in time. Its default proxy read timeout is 125 seconds, and only some plans can extend it.

Which timeouts sit between the browser and your app?#

A request passes several hops, and each can enforce a limit:

• Connect timeout: how long the proxy waits to open a TCP connection to the app. nginx defaults to 60 seconds (proxy_connect_timeout), and notes that it usually cannot exceed 75 seconds.
• Read timeout: how long the proxy waits between two reads of the app's response. nginx defaults to 60 seconds (proxy_read_timeout). It is a gap between reads, not a limit on the whole response, so a streaming response that keeps sending bytes does not trip it.
• Send timeout: the same idea for writing the request to the app, 60 seconds by default.
• Idle timeout: how long a connection can sit with no data before the balancer closes it. An AWS Application Load Balancer defaults to 60 seconds, configurable from 1 to 4000 seconds.
• Keep-alive timeout: how long a finished connection stays open for reuse. Node.js http servers default to 5000 ms.

The slowest legitimate request must fit under the smallest limit on the path. A report that takes 90 seconds works behind nothing and fails behind a 60 second proxy.

How do you tell which hop timed out?#

Measure from the outside and look at the elapsed time. A timeout is a clock, so the failure time gives it away.

curl -s -o /dev/null -w "status=%{http_code} connect=%{time_connect}s ttfb=%{time_starttransfer}s total=%{time_total}s\n" \
  https://example.com/slow-endpoint

Read the result:

Then check the other side. Is the same URL fast when you call the app directly, bypassing the balancer? If yes, the app is fine and the limit is on the path. If it is also slow, the app is the problem and raising timeouts only hides it. Your app logs should show the request arriving and how long the handler took. If the request never appears in them, it died before reaching the app.

What are the most common causes and fixes?#

From most to least common.

1. A handler that is too slow#

A report, an export, a slow query or an external API call takes longer than the proxy allows. Fix it in this order:

1. Find the slow part (query plan, upstream call, blocking CPU work).
2. Make it fast: add an index, cache the result, move work off the event loop.
3. If it is inherently long, return 202 Accepted with a job ID immediately and let the client poll or receive a webhook.

Raising the timeout is the last resort, and it only moves the wall. If you do, change it on every hop, not just one:

location / {
    proxy_pass http://app;
    proxy_connect_timeout 5s;
    proxy_read_timeout    120s;
    proxy_send_timeout    120s;
}

A short connect timeout is a good idea: if the app is not accepting connections, fail in seconds instead of waiting a minute.

2. Keep-alive mismatch (intermittent 502)#

This one confuses people because the app is healthy. The balancer keeps connections open and reuses them. If the app closes an idle connection first, the balancer can send a request down a connection that is already dead and returns a 502. AWS documents this: set the application's idle timeout larger than the load balancer's.

Node.js servers close idle connections after 5 seconds by default, which is shorter than almost every balancer. Raise it:

import http from "node:http";

const server = http.createServer((req, res) => {
  res.end("ok");
});

// Longer than the load balancer's idle timeout (60s by default on an ALB).
server.keepAliveTimeout = 65_000;
// Keep headersTimeout above keepAliveTimeout.
server.headersTimeout = 66_000;

server.listen(process.env.PORT ?? 3000);

On nginx in front of an app, the matching knob is the keepalive_timeout directive inside upstream, which defaults to 60 seconds.

3. Backend down or unhealthy (503 and 502)#

If the process crashed, ran out of memory or is listening on the wrong port, the proxy gets "connection refused" and answers 502 or 503. Check that the app binds to 0.0.0.0 and the port the proxy forwards to, not 127.0.0.1 inside a container, and read the last lines of the log for a crash. Out-of-memory kills are covered in out-of-memory errors.

4. Blocked event loop#

A single-threaded runtime that runs a long synchronous loop stops answering everything, including health checks. The balancer sees timeouts on all routes at once. Move CPU-heavy work to a worker or a queue.

5. Long-lived connections#

WebSockets and server-sent events look idle between messages, so an idle timeout kills them. Send a ping or comment line more often than the shortest idle timeout on the path. Note that an ALB does not count HTTP/2 PING frames as traffic, so send real data. See how to host a WebSocket server.

How do you prevent these errors?#

• Write down the limits. Note the timeout of every hop. A common practice is to order them client > CDN > balancer > proxy > app, so each inner layer gives up before the outer one and users get your error page, not a generic one.
• Time your endpoints. Alert on p95 and p99 latency, not the average. A p99 near your timeout is a future 504.
• Fail fast on connect. A few seconds is enough for a local connection.
• Make long work asynchronous with a job ID and status endpoint.
• Keep several backends. With more than one healthy app behind a load balancer, one crash does not take the site down. Only retry requests that are safe to repeat: a retried GET is harmless, a retried POST can charge a card twice.
• Watch the edge too. A CDN or reverse proxy in front adds its own limit, so test through the full path, not only the origin.

How to do this on Square Cloud#

On Square Cloud, a custom domain can be served by several applications at once. Traffic is distributed round-robin with passive failover (there is no active health check): if one application stops answering, requests go to another in the group. That is the "keep several backends" advice, without running a balancer of your own.

You deploy the applications and attach the same custom domain to each one. The number of applications per domain depends on the plan: up to 2 on Standard, up to 5 on Pro and up to 10 on Enterprise.

Your containers stay on, and auto-restart (on by default for dashboard deploys) brings an application back when its process exits with an error. Running out of memory stops the application until you raise its RAM and start it again, so leave some headroom. Live logs (dashboard, CLI, VS Code and the mobile app) show your process output, errors included. On Pro and Enterprise, p50/p95/p99 latency and HTTP logs show you when an endpoint is creeping toward a timeout.

The proxy in front of your app waits up to 30 seconds to connect to your application and does not cut long connections: WebSocket and SSE pass through without buffering. The rest of the checklist still applies: keep handlers fast, move long work to a job, and set your app's keep-alive timeout above the idle timeout of whatever sits in front of it. See how it works in load balancing on Square Cloud.