What is SRE, really?

The problem SRE was invented to solve

Every engineering organisation eventually hits the same wall. Developers are measured on shipping features. Operations is measured on keeping things up. Shipping changes things, and changing things breaks them, so the two groups end up with directly opposing incentives. The usual outcome is a change advisory board, a release freeze, and a quarterly argument nobody wins.

Site Reliability Engineering is Google's answer to that deadlock, published as a discipline in 2016 but practised internally since 2003. The core move is deceptively simple: stop arguing about whether to ship, and agree in advance on how much unreliability the product is allowed to have. Once that number exists, the argument becomes arithmetic.

What an SRE actually does

The title covers a wide range of jobs, and the day-to-day varies enormously between an SRE embedded in a product team and one running a shared platform. Across both, the work tends to fall into six buckets:

• Define and defend reliability targets. Choose the indicators, set the objectives, and make the data trustworthy enough that people act on it.
• Build observability. Instrumentation, metrics pipelines, log aggregation, tracing, dashboards, and the alerting that hangs off all of it.
• Engineer away toil. Automate the manual, repetitive, tactical work that scales linearly with traffic.
• Run and improve incident response. On-call rotations, incident command, postmortems, and the follow-up that actually gets done.
• Capacity and performance work. Load testing, forecasting, autoscaling policy, and cost per request.
• Production readiness review. The gate a new service passes before it is allowed to page a human at 3 a.m.

Notice that only one of those six is reactive. If your SRE team spends most of its time firefighting, the model has failed and the fix is usually organisational rather than technical.

Reliability is a product decision, not a maximum

The instinct of most engineers is that more reliability is always better. It is not, and the reason is that reliability follows a brutal cost curve. Going from 99% to 99.9% availability is usually a matter of removing single points of failure and fixing the worst deploy practices. Going from 99.99% to 99.999% means multi-region active-active, automated failover tested continuously, and a change process so conservative that feature velocity collapses.

There is also a ceiling you do not control. If the user's mobile network is available 99% of the time, the difference between a 99.99% and a 99.999% backend is invisible to them. You are spending real money to improve a number nobody can perceive.

The right target is the lowest one your users and your contracts will accept. Anything above that is budget you could have spent on the product.

The error budget, in one paragraph

If you commit to 99.9% availability over a 30-day window, you have committed to 99.9% success. The remaining 0.1% is the error budget: roughly 43 minutes of full downtime, or 1 failed request in every 1,000, or any combination that adds up. That budget is not a failure to be avoided. It is a resource to be spent, on risky deploys, on migrations, on experiments.

The policy attached to it is what makes it real. While budget remains, the team ships. When the budget is exhausted, feature work pauses and reliability work takes priority until the window rolls forward. That single rule converts every reliability conversation from a matter of opinion into a matter of arithmetic, and it is the reason SRE works. Part 3 covers the mechanics in detail.

SRE, DevOps, and platform engineering

These three overlap enough to cause endless job-title arguments. The useful distinction is what each one optimises for.

In practice one team frequently does all three, and that is fine. The reason to keep the distinction in your head is that they fail differently. A DevOps failure looks like a slow pipeline. An SRE failure looks like an unexplained outage. A platform failure looks like every team building their own snowflake.

Toil, and why it is measured

Toil has a precise definition in SRE, and it is narrower than 'work I dislike'. Work is toil when it is manual, repetitive, automatable, tactical rather than strategic, devoid of enduring value, and grows at least linearly with the size of the service.

Restarting a stuck worker by hand is toil. Manually copying a config to twelve servers is toil. Writing the controller that restarts stuck workers is not, because you do it once. Debugging a novel outage is not toil either, even though it is unpleasant, because each one teaches you something durable.

The classic guidance is to cap toil at 50% of an SRE's time. The number matters less than the act of measuring it. Tag your ticket queue, sample your week, and look at the trend. Toil that is never measured always grows, because it arrives one small task at a time and nobody notices the total.

Practice

# A crude but effective start: label your work and count it monthly.
# In your issue tracker, require one of these labels on every closed ticket.

  work-type/toil          # manual, repetitive, automatable
  work-type/engineering   # builds something durable
  work-type/incident      # unplanned, reactive
  work-type/overhead      # meetings, reviews, admin

# Then chart the ratio over time. A rising toil line is a hiring
# argument or an automation backlog, not a personal failing.

Where to start if you are starting today

1. Pick one user-facing service that matters. Not the whole estate.
2. Write down the one or two things a user would call 'working' for that service. Usually 'my requests succeed' and 'my requests are fast enough'.
3. Find out whether you can currently measure those. Most teams discover they cannot, and that discovery is the real first deliverable.
4. Set a deliberately unambitious SLO based on what the service already does, not what you wish it did.
5. Watch it for a month without attaching any consequences. Fix the measurement, then attach the policy.

That sequence takes about six weeks and it will teach you more about your system than any amount of reading. Part 2 covers step two and four properly: how to choose an indicator that reflects real user pain, and how to turn it into an objective you can defend.

Quick answers