The Site Reliability Podcast with Fexingo: SRE, Uptime, and Production Engineering

In this episode of The Site Reliability Podcast, Lucas and Luna explore how SRE teams use runbooks to standardize incident response, reduce mean time to repair, and prevent cognitive overload during outages. They break down the anatomy of a good runbook—clear triggers, step-by-step diagnostic actions, escalation paths—and contrast it with the common failure mode of stale, never-updated documentati...

In this episode, Lucas and Luna explore how Airbnb's SRE team uses a technique called traffic shifting to prevent cascading failures during large-scale releases. They break down a real incident from 2024 where a problematic database migration was caught and rolled back in minutes thanks to gradual traffic redirection. Listeners learn the specific metrics Airbnb monitors—latency p99 and error rate—...

Episode 102 of The Site Reliability Podcast explores how top SRE teams manage cognitive load during high-stakes incidents. Lucas and Luna break down the concept of cognitive load—intrinsic, extraneous, and germane—and explain why even the best runbooks fail if they overwhelm responders. The episode uses a real example from a major e-commerce platform's database failover to show how limiting active...

In this episode of The Site Reliability Podcast, Lucas and Luna explore how SRE teams build and use dependency graphs to map service interconnections and prevent cascading failures. They discuss the 2021 Fastly outage as a real-world example of a dependency chain gone wrong, and walk through how teams at companies like Google and Netflix maintain dynamic dependency maps using tools like service me...

In this milestone 100th episode, Lucas and Luna zero in on canary deployments – a key SRE strategy for rolling out software changes safely to a small subset of users before full release. They walk through a concrete example from a major online retailer that cut its mean time to recover from four hours to under seven minutes by integrating canary analysis into its continuous delivery pipeline. The...

Lucas and Luna explore how site reliability teams quantify the financial cost of incidents to make a business case for reliability spending. Using a concrete example from a mid-sized e-commerce company, they break down the incident cost calculation model that combines revenue loss, engineering overtime, customer churn, and reputational damage. They discuss how SRE teams present these numbers to le...

Episode 98 of The Site Reliability Podcast with Fexingo dives into observability pipelines — the middleware that sits between your systems and your monitoring tools. Lucas and Luna explore how teams at companies like DoorDash and Slack have used pipelines to filter, sample, and shape telemetry data before it reaches their observability backends, slashing costs by up to 70 percent without losing si...

The most resilient systems aren't built by perfect engineers — they're built by teams that can learn from failure without fear of punishment. In this episode, Lucas and Luna explore the transition from 'root cause analysis' to 'blameless postmortems' at companies like Etsy and Google. They unpack how blameless culture actually works in practice: how Etsy's 'Blameless Postmortem' framework reduced...

Every outage feels urgent, but not every issue deserves a full war room. In this episode, Lucas and Luna dive into how SRE teams use incident severity frameworks — from P0 to P5 — to classify, triage, and respond efficiently. They break down the trade-offs between speed and accuracy when assigning severity, using real examples from major outages at GitHub and Cloudflare. Learn why well-defined sev...

In episode 95 of The Site Reliability Podcast with Fexingo, Lucas and Luna explore how synthetic monitoring helps SRE teams detect issues before they impact real users. They break down the difference between synthetic and real-user monitoring, walk through a concrete example at a large e-commerce platform where synthetic checks caught a checkout failure ahead of a flash sale, and discuss how to de...

In this episode of The Site Reliability Podcast with Fexingo, Lucas and Luna break down how SRE teams adopt Incident Command Systems (ICS) from emergency services to structure their response to major outages. They walk through a real-world example of a large e-commerce platform that used ICS during a Black Friday traffic surge, explaining how the Incident Commander, Scribe, and Operations roles pr...

Episode 93 of The Site Reliability Podcast with Fexingo. Lucas and Luna dive into saturation metrics — the often-overlooked cousin of CPU and memory — and how they act as early warning signals for capacity bottlenecks. Using the real-world example of a mid-2025 incident at a major European e-commerce platform, they break down the difference between utilization and saturation, why queue depth matte...

Episode 92 of The Site Reliability Podcast with Fexingo dives into SLO burn rates — a powerful metric that tells you not just when an SLO is violated, but how fast you're burning through your error budget. Lucas and Luna walk through a real incident at a major streaming platform where a 2% error rate over 10 minutes triggered an urgent response, while the same rate over an hour wouldn't have. They...

In this episode of The Site Reliability Podcast, Lucas and Luna explore how SRE teams use Game Days — simulated incidents — to build muscle memory and improve real-world response. They break down why Netflix's Chaos Monkey was just the beginning, and how modern teams run everything from network partitions to database failovers in a controlled environment. The conversation covers the key elements o...

In this episode, Lucas and Luna dive into the concept of error budgets—a cornerstone of Site Reliability Engineering that defines how much unreliability a team can tolerate while still meeting their Service Level Objectives. They explore how error budgets help SRE teams make data-driven trade-offs between shipping new features and maintaining system stability. Using examples from Google's original...

In this episode of The Site Reliability Podcast, Lucas and Luna dive into the world of incident metrics — not just DORA or SLOs, but the specific numbers that help SRE teams get faster and better at incident response. They discuss mean time to acknowledge, mean time to resolve, and the controversial metric of mean time between failures, using real examples from a major cloud provider's 2023 outage...

Episode 88 of The Site Reliability Podcast with Fexingo dives into how SRE teams can cut cloud costs without sacrificing reliability. Lucas and Luna discuss the rise of FinOps, the hidden waste in over-provisioned resources, and how Google, Netflix, and Airbnb use committed use discounts, spot instances, and right-sizing to save millions. Learn the concrete metrics—like cost per transaction and id...

Episode 87 of The Site Reliability Podcast explores toil budgets — a practice Google SRE pioneered to cap repetitive, non-valuable operational work. Lucas and Luna break down why Google set a 50% toil limit, how to measure toil versus engineering, and why companies like Etsy and Netflix use toil budgets to protect innovation time. They also discuss common pitfalls: treating all toil equally and fo...

In this episode of The Site Reliability Podcast, Lucas and Luna explore how SRE teams deliberately inject small, controlled failures into production not to break things but to build collective learning. They dissect the approach used by a major payments company that runs weekly 'structured fail' exercises where engineers intentionally trigger a known-category incident (latency spike, partial data...

In Episode 85 of The Site Reliability Podcast, Lucas and Luna explore how SRE teams turn post-incident reviews into actionable system improvements. They focus on a real-world case: a major streaming service's 2023 outage caused by a cascading failure in their content delivery network. The hosts break down the review process, from timeline reconstruction to root cause analysis to implementing preve...

In this episode of The Site Reliability Podcast, Lucas and Luna dive into capacity planning for SRE teams — the proactive discipline that keeps systems running when traffic spikes. Using the example of a major streaming platform's 2024 holiday season surge, they break down how capacity planning differs from simple scaling, why it's part of reliability engineering, and how teams use traffic forecas...

Lucas and Luna dive into chaos engineering, using Netflix's Chaos Monkey and the Simian Army as the prime example. Lucas explains how Netflix intentionally broke its own systems in production to uncover weaknesses before they caused real outages, citing the tool's origin story from 2011 and its evolution into a formal discipline. Luna challenges the notion that chaos experiments are too risky for...

Episode 82 of The Site Reliability Podcast examines how cost of delay — a concept borrowed from product development — helps SRE teams decide which reliability projects to tackle first. Lucas and Luna walk through a real example from a mid-sized fintech company that used cost of delay to justify migrating from a legacy database to a distributed SQL solution. The episode explains how to calculate co...

Lucas and Luna dive into latency budgets — a less-discussed SRE tool that maps acceptable delay across each microservice in a user request chain. They use the example of a social media app's photo upload feature: if the overall latency SLO is 500 milliseconds, the team allocates 50 ms to the auth service, 200 ms to the image processing service, and so on. Lucas explains how Google's internal SRE t...

In episode 80 of The Site Reliability Podcast, Lucas and Luna dive into the practical world of runbooks — the step-by-step guides that SRE teams use to respond to incidents faster and more consistently. They explore how runbooks reduce cognitive load during high-stress outages, why documenting the 'why' behind each step prevents dangerous cargo-culting, and how a major streaming service cut its me...