A site reliability engineer exists to keep systems running well enough that the\nbusiness and its users can trust them, while still letting those systems change\nfast. The discipline was born at Google from a refusal to choose between two bad\noptions: a fragile system that ships features quickly, or a frozen system that\nnever breaks because nothing ever moves. An SRE makes reliability a measurable,\nengineered property — something you budget, design for, and trade against\nvelocity on purpose — rather than a thing you hope for and apologize for when it\nfails.
\n","wordCount":92},{"heading":"Core Mission","id":"core-mission","markdown":"Run services at a deliberately chosen level of reliability — high enough that\nusers don't notice, low enough that the team keeps shipping — by treating\noperations as a software problem.","html":"Run services at a deliberately chosen level of reliability — high enough that\nusers don't notice, low enough that the team keeps shipping — by treating\noperations as a software problem.
\n","wordCount":29},{"heading":"Primary Responsibilities","id":"primary-responsibilities","markdown":"The visible work is responding to pages, but the actual work is making pages\nrare. An SRE defines what \"working\" means in numbers (SLIs and SLOs), measures\nit, and uses the gap between target and reality — the error budget — to govern\nhow aggressively the product team can ship. They build the automation that\nremoves humans from the routine path: deploys, rollbacks, capacity changes,\nfailovers. They design systems to degrade gracefully and recover automatically.\nThey run incidents as incident commander, write blameless postmortems, and turn\neach outage into a permanent fix. They do capacity planning so the service\ndoesn't fall over under entirely predictable load. And they hard-cap the time\nspent on toil so the rest can go to engineering that makes toil disappear.","html":"The visible work is responding to pages, but the actual work is making pages\nrare. An SRE defines what "working" means in numbers (SLIs and SLOs), measures\nit, and uses the gap between target and reality — the error budget — to govern\nhow aggressively the product team can ship. They build the automation that\nremoves humans from the routine path: deploys, rollbacks, capacity changes,\nfailovers. They design systems to degrade gracefully and recover automatically.\nThey run incidents as incident commander, write blameless postmortems, and turn\neach outage into a permanent fix. They do capacity planning so the service\ndoesn't fall over under entirely predictable load. And they hard-cap the time\nspent on toil so the rest can go to engineering that makes toil disappear.
\n","wordCount":124},{"heading":"Guiding Principles","id":"guiding-principles","markdown":"- **Reliability is a feature with a budget, not an absolute.** 100% is the wrong\n target for almost everything; it's infinitely expensive and users can't tell\n past a point. Pick a number and defend it.\n- **Hope is not a strategy.** If you can't measure it, you can't promise it.\n Every claim must trace to an SLI.\n- **Toil is the enemy.** Manual, repetitive, automatable work that scales with\n the service is a tax on the future. Cap it, then engineer it away.\n- **Blameless or it's worthless.** People who fear punishment hide information,\n and hidden information is how the next outage gets worse.\n- **Make the system boring.** Excitement in operations means something is wrong.\n Aim for days where nothing happens.\n- **Automate yourself out of the loop, but keep the human override.** The robot\n handles the common case; the human handles what the robot didn't foresee.\n- **The error budget belongs to everyone.** When it's spent, you stop shipping\n risky changes — not as punishment, as physics.","html":"SREs sit at the seam between development and operations, and the relationship is\nthe job. With product and software engineers, the SRE negotiates SLOs and error\nbudgets, pushes reliability requirements left into design, and reviews\nproduction-readiness before launch. With security engineers, they share the\nincident-response muscle; with platform and DevOps teams, the paved road. The\nrecurring tension is ownership: SRE works best when development teams carry their\nown pager and SRE provides the platform, the standards, and the hard problems —\nnot when SRE becomes a dumping ground for everyone else's operational debt. The\nhealthiest arrangement is a consulting/embedding model with the right to hand a\nservice back if it doesn't meet the reliability bar.
\n","wordCount":117},{"heading":"Ethics","id":"ethics","markdown":"SREs hold a quiet power: the on-call engineer's judgment at 3 a.m. can keep a\nhospital's records reachable or a payment system honest. The duties that follow:\ntell the truth in postmortems even when it implicates a process you built;\nrefuse to paper over systemic risk with heroics that hide the real fragility;\nresist pressure to set SLOs the team knows it can't meet just to win a deal;\nprotect the humans on the rotation from burnout, because a fried on-call is itself\na reliability risk; and be honest about what redundancy buys versus what's\ntheater. When the business wants to advertise availability it can't deliver, the\nSRE's job is to say so.","html":"SREs hold a quiet power: the on-call engineer's judgment at 3 a.m. can keep a\nhospital's records reachable or a payment system honest. The duties that follow:\ntell the truth in postmortems even when it implicates a process you built;\nrefuse to paper over systemic risk with heroics that hide the real fragility;\nresist pressure to set SLOs the team knows it can't meet just to win a deal;\nprotect the humans on the rotation from burnout, because a fried on-call is itself\na reliability risk; and be honest about what redundancy buys versus what's\ntheater. When the business wants to advertise availability it can't deliver, the\nSRE's job is to say so.
\n","wordCount":116},{"heading":"Scenarios","id":"scenarios","markdown":"**Error budget exhausted mid-quarter.** A team has burned through its 0.1%\navailability budget by week three, mostly from rushed deploys. The product\nmanager wants to ship a launch anyway. The expert SRE doesn't argue about this\nlaunch in isolation; they point to the error-budget policy agreed months ago: no\nrisky changes until the budget recovers. The conversation shifts from \"should we\nship?\" to \"how do we restore reliability fastest?\" — which surfaces that two\noutages came from a missing canary. The launch slips a week, the canary gets\nbuilt, and the policy holds because it wasn't invented in the heat of the moment.\n\n**A latency-driven cascade.** A downstream database gets slow — not down, slow.\nUpstream services hit their timeouts, retry, and the retries pile onto the\nalready-struggling database, driving it fully over. The on-call's instinct isn't\nto restart the database; it's to stop the amplification: load shedding, a circuit\nbreaker so callers fail fast, jitter on the backoff. Service stabilizes within\nminutes at degraded capacity. The postmortem's real fix isn't \"make the database\nfaster\" — it's that retries had no backoff, so the system attacked itself.\n\n**Deciding an SLO for a new service.** Engineering proposes 99.99%. The SRE asks\nwhat users actually need: this is an internal batch-reporting service whose\nconsumers read it once a day. Four nines would demand multi-region failover and\non-call coverage that costs more than the service is worth. They set 99.5%, write\nit down, and redirect the saved effort to a user-facing service where the nines\nare felt. Reliability spent where it matters.","html":"Error budget exhausted mid-quarter. A team has burned through its 0.1%\navailability budget by week three, mostly from rushed deploys. The product\nmanager wants to ship a launch anyway. The expert SRE doesn't argue about this\nlaunch in isolation; they point to the error-budget policy agreed months ago: no\nrisky changes until the budget recovers. The conversation shifts from "should we\nship?" to "how do we restore reliability fastest?" — which surfaces that two\noutages came from a missing canary. The launch slips a week, the canary gets\nbuilt, and the policy holds because it wasn't invented in the heat of the moment.
\nA latency-driven cascade. A downstream database gets slow — not down, slow.\nUpstream services hit their timeouts, retry, and the retries pile onto the\nalready-struggling database, driving it fully over. The on-call's instinct isn't\nto restart the database; it's to stop the amplification: load shedding, a circuit\nbreaker so callers fail fast, jitter on the backoff. Service stabilizes within\nminutes at degraded capacity. The postmortem's real fix isn't "make the database\nfaster" — it's that retries had no backoff, so the system attacked itself.
\nDeciding an SLO for a new service. Engineering proposes 99.99%. The SRE asks\nwhat users actually need: this is an internal batch-reporting service whose\nconsumers read it once a day. Four nines would demand multi-region failover and\non-call coverage that costs more than the service is worth. They set 99.5%, write\nit down, and redirect the saved effort to a user-facing service where the nines\nare felt. Reliability spent where it matters.
\n","wordCount":268},{"heading":"Related Occupations","id":"related-occupations","markdown":"The site reliability engineer is a software engineer who optimizes for the\nsystem's survival over its features, sharing the same code fluency aimed at a\ndifferent objective. DevOps engineers overlap heavily — both automate the path to\nproduction — but SRE is defined by SLO-driven operation rather than the delivery\npipeline. Systems administrators are the operational ancestor, before operations\nbecame a software-engineering problem. Cloud architects design the redundant\nsubstrate SREs keep alive, and security engineers share the incident-response\ndiscipline applied to a different threat.","html":"The site reliability engineer is a software engineer who optimizes for the\nsystem's survival over its features, sharing the same code fluency aimed at a\ndifferent objective. DevOps engineers overlap heavily — both automate the path to\nproduction — but SRE is defined by SLO-driven operation rather than the delivery\npipeline. Systems administrators are the operational ancestor, before operations\nbecame a software-engineering problem. Cloud architects design the redundant\nsubstrate SREs keep alive, and security engineers share the incident-response\ndiscipline applied to a different threat.
\n","wordCount":85},{"heading":"References","id":"references","markdown":"- *Site Reliability Engineering* — Beyer, Jones, Petoff, Murphy (Google)\n- *The Site Reliability Workbook* — Beyer et al.\n- *Designing Data-Intensive Applications* — Martin Kleppmann\n- *Release It!* — Michael Nygard\n- USENIX SREcon proceedings","html":"