Why is my server or app slow? CPU steal, memory and I/O

"The server is slow" is a symptom, not a diagnosis. Behind it there is either a saturated resource (CPU, memory, disk, network) or a limit imposed from outside (CPU shared with neighbors, a container quota). Sometimes it is not the server at all: it is a query without an index or a loop blocking the process.

Upgrading without measuring is the most expensive mistake here. The investigation below goes from the cheapest check to the hardest, with the commands that tell each cause apart.

How do I tell CPU, memory, disk and network apart?#

Start with what the machine reports about itself. On a Linux server, three commands answer most questions:

top            # "%Cpu(s)" line: us, sy, wa (I/O) and st (steal)
vmstat 1 5     # r (run queue), si/so (swap), wa, st, one line per second
free -m        # available memory and swap usage

Read the output like this:

• High us/sy and run queue r above the core count: CPU is saturated.
• Non-zero si/so: the system is swapping, so memory is short.
• High wa: processes are waiting on disk.
• High st: CPU stolen by the hypervisor (next section).
• Everything low but the app is slow: time is spent waiting on something external (database, API, network) or in code that does not burn much CPU.

What are CPU steal and overselling?#

On a VPS, your "server" is a virtual machine, and several VMs share the same physical cores. Linux's steal field is, per the proc_stat man page, time spent in other operating systems when running in a virtualized environment: your VM had work to do and the hypervisor gave the core to another guest.

Overselling (or overcommit) is when a provider allocates more vCPUs to customers than there are physical cores, betting that not everyone is busy at once. It is common on cheap plans, but when neighbors get busy together, your steal rises. Some instance types are also explicitly "burstable": AWS describes its T instances as a baseline CPU with the ability to burst above it, not full CPU all the time.

To diagnose it:

1. Run vmstat 1 60 during the slowdown and watch the st column.
2. If st stays high repeatedly while your user CPU (us) is low, the bottleneck is not your code.
3. Repeat at different times. Steal that disappears at night and returns in the evening points to a noisy neighbor.

The fix is not in your app: change instance type (dedicated CPU), region or provider. Before that, open a support ticket with the numbers, which is the evidence support teams ask for.

What is CPU throttling in a container?#

Containers have another cause that looks like steal but is not. A container gets a CPU quota: in cgroup v2, the cpu.max file sets how much CPU time the group may use per period (the default is max 100000, meaning no limit). Past the quota, the kernel pauses the processes until the next period, which shows up as latency spikes.

Inside the container, check:

cat /sys/fs/cgroup/cpu.max    # e.g. "50000 100000" = half a core
cat /sys/fs/cgroup/cpu.stat   # nr_throttled and throttled_usec growing = throttling

If nr_throttled grows while the app is slow, the container CPU limit is the bottleneck. Raise the limit or reduce the work per request. Unlike steal, this one is yours to fix.

Is the app running out of memory?#

Running out of memory usually shows up in one of two ways. The first is slowness from swapping: the system moves pages to disk and everything crawls, and vmstat shows active si/so. The second is the process dying: the kernel kills processes that exceed the container limit and the oom_kill counter in memory.events goes up.

cat /sys/fs/cgroup/memory.events   # oom_kill > 0: something was killed for lack of memory

A classic sign is an app that gets slower over hours and recovers after a restart: that is a memory leak or an unbounded cache. See how to measure the peak and size with headroom in How much RAM and CPU does my app need?.

What if CPU and memory are fine?#

Then the time is spent waiting. The suspects, in order of frequency:

1. Database: a query without an index, N+1 (one query per list item) or a connection opened per request. Turn on your database's slow query log and time each query.
2. External API: a slow third-party call inside the request path. Add a timeout and, when possible, move the call off the critical path.
3. Disk: high wa in vmstat, or iostat -x 1 (from the sysstat package) showing a saturated disk. Excessive logging, large temp files and databases on slow disks are the usual culprits. The man page warns that the iowait value is unreliable, so treat it as a hint, not proof.
4. Distance: network latency between the app and the database, or between server and user. Keep the app and database in the same datacenter, and static content benefits a lot from a CDN.

How do I find the bottleneck inside the code?#

In Node.js the typical symptom is a blocked event loop: a heavy loop or synchronous call stops the process from serving any other request. The perf_hooks module measures that delay directly:

// event-loop.js: prints event loop delay every 5 seconds
const { monitorEventLoopDelay } = require("node:perf_hooks");

const h = monitorEventLoopDelay({ resolution: 20 });
h.enable();

setInterval(() => {
  const ms = (ns) => (ns / 1e6).toFixed(1);
  console.log(
    `event loop: mean=${ms(h.mean)} ms p99=${ms(h.percentile(99))} ms max=${ms(h.max)} ms`,
  );
  h.reset();
}, 5000);

Values are in nanoseconds, hence the division by 1e6. With free CPU, a p99 of hundreds of milliseconds means code is blocking the loop. In Python, the equivalent is a profiler such as cProfile run for a few minutes under load. The pattern is the same: measure where the time goes before changing any line.

Also log the duration of every request. With duration and route on each line the bottleneck often shows itself, and reading your app logs is the next step when the numbers alone do not explain it.

What is the right order to investigate?#

1. Confirm the symptom with a number: response time per route, not "feels slow".
2. top/vmstat: CPU, steal, swap, iowait.
3. Cgroup files (cpu.stat, memory.events) if you are in a container.
4. Database and external APIs: slow queries and timeouts.
5. A profiler on the code.
6. Only then change plan or server, and measure again to confirm it improved.

If steal is the cause, the choice between a VPS and a platform changes: see VPS, cloud or hosting platform for what each model isolates.

How to do this on Square Cloud#

On Square Cloud, applications run in isolated containers that are always on, with no cold start and no sleep mode, in a Tier 3 datacenter in New York with Intel Dual Xeon 3 GHz+ processors and NVMe storage. You follow real-time logs in the dashboard, the CLI, VS Code and the mobile app, and CPU, RAM and network metrics are recorded every 5 minutes and kept for 24 hours.

If memory is the cause, you change each application's RAM in settings or through the API. The plan's RAM is a pool shared across your applications, with a minimum of 256 MB per background application and 512 MB per web application. The plan's vCPU is a per-environment ceiling, calculated from the RAM you allocate.

Database hosting (from Standard) sits in the same datacenter as the applications, with about 1 ms average latency, which rules out app-to-database distance as a suspect. On Pro and Enterprise, analytics shows p50, p95 and p99 latency plus HTTP logs, so you can find the slow routes.

There is no autoscaling: you decide when to add RAM or spread load with load balancing, by putting the same custom domain on several applications (from the Standard plan up). See the logs, metrics and scaling features.