Capacity planning and SLOs

Everything in this part — the decomposed latency, the tail, the knee, the trade-offs — becomes actionable only when you turn it into two commitments: an SLO (the performance target you promise) and a capacity plan (the resources provisioned to keep it). This closing chapter connects them. An SLO without capacity to back it is a wish; capacity without an SLO is a guess. Together they're how a serving system moves from "it seems fast enough" to a promise you can operate against.

What you will understand by the end

• What an SLO is and why it's written on the tail.
• What capacity planning is and how the knee feeds it.
• Why you provision for peak load with headroom, not average.
• How SLOs and capacity close the loop with the rest of the book.

SLOs: the performance promise

A service level objective is a specific, measurable performance target you commit to — typically a tail-latency bound and a throughput or availability target: "p99 end-to-end latency < 2s at up to N requests/sec." It's written on the tail because that's what users feel and what saturation attacks first; a mean-based SLO would stay green through a tail collapse. The SLO makes "fast enough" concrete, gives alerts and release gates something to enforce, and — crucially — states the target at a specified load, because latency is meaningless without the concurrency it holds at.

Capacity planning: provisioning to keep the promise

Capacity planning is deciding how much hardware (or hosted quota) you need to meet the SLO at your expected load. It's driven directly by the knee you load-tested: the knee is each unit's usable capacity within the SLO, so the plan is roughly expected peak load ÷ per-unit capacity at the SLO, plus headroom. The queueing lesson sets the headroom: because latency runs away near capacity, you provision to run below the knee, not at it.

   units needed ≈  ── peak load (with burst)──  +  headroom
                    per-unit capacity at SLO
                    (the knee from load testing)

The pieces come together

Capacity planning is where the whole part converges. The knee (from load testing) gives per-unit capacity; the tail and queueing set the SLO and the headroom; the trade-off triangle fixes the configuration (and thus the knee and cost) you're provisioning; and cost per success tells you whether the plan is economically sound. And it doesn't end at Part 7: the SLO becomes a dashboard and alert in production, its breaches become incidents, and demand drift triggers re-planning. Capacity and SLOs are where offline measurement becomes a running, monitored commitment.

Mental model

An SLO is a promise on the tail at a stated load ("p99 < X at Y req/sec"); capacity planning provisions to keep it — roughly peak-load-with-burst ÷ per-unit capacity at the SLO (the load-tested knee), plus headroom below the knee because queueing runs away near it. Provision for peak, not average; re-plan as demand drifts. This is where the part's measurements become a running commitment.

Common mistakes

• SLOs on the mean, or with no load figure. The tail is what users feel and what saturates first; latency needs a load to be meaningful.
• Provisioning for average load. Peak and bursts violate the SLO the average hides; plan for peak-plus-burst.
• Provisioning at the knee, not below it. Queueing runs away near capacity; you need headroom for bursts and growth.
• Treating capacity as one-time. Demand grows and drifts; capacity planning is continuous.

Practical guidance

• Write SLOs as tail targets at a stated load ("p99 end-to-end < 2s at N req/sec"), and enforce them via alerts and gates.
• Derive capacity from the load-tested knee: peak-load-with-burst ÷ per-unit capacity at SLO + headroom, provisioning below the knee.
• Provision for peak and burst, not average, and revisit as demand drifts — it's a continuous process.
• Check the plan against cost per success so it's economically sound, not just fast enough.

Summary

• An SLO is a performance promise on the tail at a stated load; it makes "fast enough" concrete and enforceable.
• Capacity planning provisions to keep the SLO: peak-with-burst ÷ per-unit capacity at SLO (the knee) + headroom, run below the knee.
• Provision for peak, not average, and re-plan as demand drifts — capacity is continuous.
• This is where the part's measurements (knee, tail, trade-offs, cost) become a running, monitored commitment, handing off to production observability.

Knowledge check

Your load test shows each GPU serves ~50 concurrent requests within your p99 SLO. Expected peak is 400 concurrent requests, with occasional bursts to 550. How many GPUs do you provision, and why not fewer?

Start from per-unit capacity at the SLO (50) and the peak with burst (550, not the 400 average): 550 ÷ 50 ≈ 11 units at the knee — but you provision below the knee for headroom, since queueing runs away near capacity and you need slack to absorb bursts and demand growth, so round up to ~12–14 GPUs. You don't provision for the 400 average (that guarantees SLO breaches during the 550 bursts, exactly when load is highest and most visible), and you don't provision at the knee (no slack for bursts → tail latency explodes). Provision for peak-plus-burst with headroom, and re-plan as demand drifts upward.

Why must an SLO specify a load level, e.g. "p99 < 2s at 100 req/sec," rather than just "p99 < 2s"?

Because latency is a function of load — the same system easily meets p99 < 2s at low concurrency and blows past it near saturation, so a latency target without a stated load is unmeasurable and unprovisionable: you can't tell whether it's being met (met at what load?) or size capacity to keep it (capacity for what demand?). Pinning the load makes the SLO a concrete promise ("this fast, up to this much traffic") that you can verify, alert on, and turn into a capacity number via the knee. Latency and load are inseparable; an SLO must name both.

Related chapters

• Throughput and concurrency — the knee that is per-unit capacity
• Queueing and saturation — why you provision with headroom below the knee
• Median, p95 and p99 — why SLOs are written on the tail
• Dashboards, alerts and incident investigation — where the SLO becomes a live commitment