SLI, SLO, SLA: the reliability contract

Three words that get used interchangeably and should not be

Almost every reliability programme starts with someone confidently misusing these three terms in a planning meeting. They form a chain: the indicator is the measurement, the objective is the internal target for that measurement, and the agreement is the external promise about it.

What makes an SLI good

An SLI should answer one question: is the service doing its job for the user right now? That rules out most of what a typical monitoring system collects. CPU utilisation is not an SLI. Neither is pod count, queue depth, or JVM heap. Those are causes. An SLI measures the effect the user experiences.

The strongest formulation, and the one used by nearly every mature implementation, is a ratio of good events to valid events:

The pattern

SLI = good events / valid events

# Availability:  successful requests / all requests that reached us
# Latency:       requests served faster than 300ms / all requests
# Freshness:     records processed within 5 min of arrival / all records
# Correctness:   records with the right output / all records processed
# Coverage:      rows successfully ingested / rows submitted

Framing every SLI this way pays off later: it makes the error budget trivially computable, it makes latency and availability comparable, and it makes burn-rate alerting a single reusable rule template instead of a bespoke query per service.

Decide what counts as 'valid' before you decide what counts as 'good'

This is where most first attempts go wrong, and the resulting SLO is either impossible to hit or impossible to miss. You must be explicit about the denominator.

• Are 4xx responses failures? Usually not. A 400 means the client sent something invalid; that is the client's bug, and counting it against you means a single broken integration can burn your budget. But a 429 from your own overload protection is arguably yours, and a 404 on a resource that should exist certainly is.
• Are health check and probe requests included? They should not be. They are high-volume and always succeed, so they dilute the ratio and hide real failures.
• Are requests during planned maintenance included? Decide once, write it down, and implement it consistently.
• Are bot and scanner requests included? If a scanner hits 10,000 nonexistent paths, your error rate should not move.
• Is a request that times out at the client but succeeds at the server good or bad? It is bad. This is the argument for measuring as close to the user as you can.

Where you measure changes what you measure

The same nominal SLI produces very different numbers depending on the observation point.

For most teams the pragmatic answer is: measure at the ingress or service mesh as the system of record, and keep client-side RUM alongside it as a reality check. When the two disagree badly, the gap itself is the finding.

Writing the objective: target, window, and window type

An SLO needs three components. Missing any one of them makes it unenforceable.

The target

Start from measured reality, not aspiration. Pull four weeks of history, compute what the SLI actually was, and set the target slightly above the current typical performance. If you have been running at 99.5%, a 99.9% target is a project, not an objective. Set 99.5%, hold it, then tighten it deliberately.

The window

Twenty-eight or thirty days is the standard for the headline SLO. Shorter windows are twitchy and punish single incidents disproportionately. Longer windows hide sustained degradation for too long. Use a short window in addition, not instead, when you need faster feedback.

Rolling or calendar

A rolling window looks at the trailing 30 days continuously and is what you should alert on. A calendar window resets on the first of the month and is what finance and customers understand. Most teams report on calendar and alert on rolling. Just be clear which one a given number refers to.

SLO definition template

Service:        checkout-api
SLI:            proportion of valid requests served successfully
Valid events:   all HTTP requests to /api/v1/checkout/* excluding /healthz
Good events:    HTTP status < 500 AND served in < 400ms
Measured at:    ingress-nginx access logs
SLO:            99.5% of valid requests are good, over a rolling 28 days
Error budget:   0.5% = ~3h 21m of total failure, or 5 in every 1,000 requests
Policy:         see error-budget-policy.md
Owner:          checkout team
Reviewed:       quarterly

Implementing it in Prometheus

The mechanics are straightforward once the definition is fixed. The important habit is to build the SLI as a recording rule rather than computing it ad hoc in dashboards, so that every dashboard, alert, and report uses exactly the same arithmetic.

prometheus/rules/slo-checkout.yml

groups:
  - name: slo-checkout-api
    interval: 30s
    rules:
      # Denominator: all valid requests
      - record: sli:checkout_requests:rate5m
        expr: |
          sum(rate(http_request_duration_seconds_count{
            job="checkout-api", route!="/healthz"
          }[5m]))

      # Numerator: fast AND not a server error.
      # Note the subtraction: bucket counts include errors, so remove them.
      - record: sli:checkout_requests_good:rate5m
        expr: |
          sum(rate(http_request_duration_seconds_bucket{
            job="checkout-api", route!="/healthz", le="0.4"
          }[5m]))
          -
          sum(rate(http_request_duration_seconds_count{
            job="checkout-api", route!="/healthz", code=~"5.."
          }[5m]))

      # The SLI itself
      - record: sli:checkout_availability:ratio5m
        expr: |
          sli:checkout_requests_good:rate5m
          /
          sli:checkout_requests:rate5m

To report attainment over the full window, aggregate the same numerator and denominator across 28 days rather than averaging the ratio. Averaging ratios weights quiet nights equally with peak traffic and will flatter you:

28-day attainment

# CORRECT - ratio of the sums, traffic-weighted
sum_over_time(sli:checkout_requests_good:rate5m[28d])
/
sum_over_time(sli:checkout_requests:rate5m[28d])

# WRONG - mean of the ratios, weights a 3am hour the same as noon
avg_over_time(sli:checkout_availability:ratio5m[28d])

The mistakes that show up in almost every first attempt

• Too many SLOs. Two or three per user-facing service. If everything has an SLO, nothing does, and nobody can tell you which one is on fire.
• SLOs on internal components nobody uses directly. A cache hit-rate SLO is a performance metric wearing a costume. Reserve SLOs for things with consumers.
• Targets set by aspiration. Picking 99.99% because it sounds serious guarantees a permanently exhausted budget, which trains everyone to ignore it.
• Averaging away the pain. A single global SLO can look healthy while one region, one tenant, or one endpoint is completely broken. Consider per-tier or per-region SLOs for anything with meaningfully different traffic.
• No owner and no review. An SLO with no named owner drifts out of date within two quarters and quietly stops being trusted.
• Alerting directly on the SLO threshold. 'Page when availability drops below 99.9%' fires constantly during low traffic and misses slow burns. Part 3 fixes this properly.

Quick answers