This is Fine! A podcast about resilience engineering and software

Aerobarrier is super cool if you want to check out how it works: https://glbipro.com/aerobarrier-2/ Colette isn’t drinking alcohol, it’s a delicious herbal mocktail by Casamara Club: https://www.casamaraclub.com/products/como (Maybe we can get them to sponsor us someday) On the passive house that survived the Palisades fire: https://www.reddit.com/r/Damnthatsinteresting/comments/1hy22ui/house_desi...

The new website is live! thisisfinepod.com You can find John Allspaw at Adaptive Capacity Labs: https://www.adaptivecapacitylabs.com Mike McGill, the skateboarder: https://en.wikipedia.org/wiki/Mike_McGill Annie Duke’s Thinking in Bets, referenced by our question-asker is a great one: https://bookshop.org/p/books/thinking-in-bets-making-smarter-decisions-when-you-don-t-have-all-the-facts-annie-duk...

Mitchell Hashimoto’s post on leaving Github: https://mitchellh.com/writing/ghostty-leaving-github The Reddit post on Github’s availability historically (that we find questionable): https://www.reddit.com/r/github/comments/1rnvhs9/githubs_historic_downtime_scraped_and_plotted/ A reminder, the Messy 9 are: congestion, cascade, conflict, lag, saturation, friction, tempo, surprise, tangles We have som...

Colette’s talk at SRECon intro: https://www.usenix.org/conference/srecon26americas/presentation/alexander Clint’s talk at SRECon intro: https://www.usenix.org/conference/srecon26americas/presentation/byrum Dan Slimmon is an excellent engineer (per Clint’s shoutout) and ALSO an excellent podcast creator/host: https://techblows.net/ Michelle Brush’s Keynote summary is here: https://www.usenix.org/co...

You can find the 2025 DORA Report here: https://dora.dev/research/2025/dora-report/ Read more of Fred’s work/opinions here: https://ferd.ca/ If you want to know more about Lund’s Human Factors and Systems Safety program, you can read here: https://www.humanfactors.lth.se/ DORA has some good writeups of generative leadership and Westrum’s model here: https://dora.dev/capabilities/generative-organiz...

The Keewenaw snow gauge that Colette mentioned is a tourist attraction. If you want to see where measurements are at for the season you can find them here: https://www.pasty.com/snow/ The paper we’re talking about today can be found here: https://www.sciencedirect.com/science/article/abs/pii/S0003687020301903 If you want to know more about SNAFU Catchers, you can see their website here: https://ww...

Colette mentioned Menlo Innovations https://menloinnovations.com/ and Atomic Object https://atomicobject.com/ who both build custom software for folks. The CEO of Menlo is Richard Sheridan who wrote Joy, Inc. - https://bookshop.org/p/books/joy-inc-how-we-built-a-workplace-people-love-richard-sheridan/7677689?ean=9781591847120&next=t Chad Todd’s thesis on Handovers in Software Operations is wor...

Special thanks to John Allspaw, Sheeri Cabral, Martin Smith, and David Woods for joining us! Ben Affleck’s been making the promo rounds, but the specific convo we reference is recapped here: https://www.moviemaker.com/ben-affleck-ai-explains/ The Messy 9 are: congestion cascade conflict lag saturation friction tempo surprise tangles Dave’s been doing a set of videos on Resilience Engineering, some...

If you’re feeling like you need to do more to respond to our moment: Lots of place to donate to in the twin cities are listed here: https://mspmag.com/arts-and-culture/general-interest/ice-minnesota-support-immigrant-communities-fundraisers-food-drives-trainings/ You can always find mutual aid networks in your own area, including immigrant aid networks https://immigrantdefensenetwork.org/ does goo...

Seriously though, can’t wait to gtfo of this year. Palisades fire links: https://www.nbclosangeles.com/investigations/anonymous-letter-demands-independent-palisades-fire-investigations/3800442/ https://internationalfireandsafetyjournal.com/palisades-fire-report/ https://www.latimes.com/california/story/2025-12-20/lafd-report-on-palisades-fire-was-watered-down-in-editing-process-records-show Corey...

Mentioned multiple times, Em Ruppe’s amazing talk on incident severity: https://www.usenix.org/conference/srecon24americas/presentation/ruppe We talk about the RIS Slack sometimes - you can join us in the slack, by joining the Foundation here: https://resilienceinsoftware.org/ Please ask us a question at thisisfinepod.com  

The writeup on the AWS outage from AWS themselves, if you haven’t seen it: https://aws.amazon.com/message/101925/ Dave’s department at OSU, Cognitive Systems Engineering: https://ise.osu.edu/human-systems-integration/cognitive-systems-engineering is a part of the larger Integrated Systems Engineering school: https://ise.osu.edu/human-systems-integration   Dave was talking early on about the d...

It’s Spamton G (not J) Spamton, Clint! Get hip to the game characters!  https://deltarune.fandom.com/wiki/Spamton There are a couple of incident command trainers out there who tend to get recommended in the tech world (that we know of): https://www.blackrock3.com/ and Great Circle: https://greatcircle.com/im/

A history of the 5 whys and root cause analysis from papers Some critiques of the 5 whys: From John Allspaw: https://www.oreilly.com/radar/the-infinite-hows/ From Alan J Card: https://qualitysafety.bmj.com/content/26/8/671 James Reason and the Swiss Cheese Model:  https://pmc.ncbi.nlm.nih.gov/articles/PMC8514562/ James Reason’s book Human Error: https://bookshop.org/p/books/human-error/9e06d8...

More robustness than resilience, but worth repeating that you should always check your earthquake go-bag: https://www.earthquakeauthority.com/blog/2019/how-to-make-an-earthquake-emergency-kit Clint did ASA 103: https://americansailing.com/learn-to-sail/certifications/asa-103-coastal-cruising/ Since this is a science podcast, there is a scientific reason people get emotional on airplanes: https://w...

Lorikeets are pretty: https://en.wikipedia.org/wiki/Rainbow_lorikeet You think Colette’s kidding about the kangaroo? https://www.youtube.com/watch?v=DQjHVRHXbc8   The Mackinac Bridge is long: https://en.wikipedia.org/wiki/Mackinac_Bridge Michelle’s Blog post: https://resilienceinsoftware.org/news/1288714 DORA has some good writing on Westrum’s cultural models if you’re wondering about it: htt...

Corn sweat is a real thing: https://www.scientificamerican.com/article/humidity-from-corn-sweat-intensifies-extreme-heat-wave-in-midwest-u-s/ Also, plugging Tajin here, because: https://en.wikipedia.org/wiki/Taj%C3%ADn_seasoning Wikipedia tells me Tajin is Mexican. I dunno, Clint. Beaumaiden report, for those that didn’t listen to the prior episode where we mentioned it: https://dmaib.com/reports/...

A huge thanks to our panelists: ⁠John Allspaw⁠ ⁠Jed Needle⁠ ⁠Chad Todd⁠ RISF and TiF will host a live follow up to this episode on July 31st! You can sign up here: ⁠https://resilienceinsoftware.org/networks/events/133948⁠ If you’re interested in Lund’s Masters of Science program in Human Factors and Systems Safety, or any of their learning labs, you can check out more info here: ⁠https://www.human...

Dave Wood’s Talk at SRECon 25 was on Complexification and SRE: https://www.youtube.com/watch?v=lmBvUJnGUX4 Jens Rasmussen’s model - Is really well explained by Richard Cook’s talk at Velocity: https://www.youtube.com/watch?v=PGLYEDpNu60&t=3s Lorin’s blog also has a good summary: https://surfingcomplexity.blog/2021/05/31/transgressing-the-boundaries-rasmussen-and-woods/ And finally, Jens Rasmus...

You can register for the After-the-Episode chat with Andrew at https://resilienceinsoftware.org/networks/events/129997 Tickets are free for members, $10 for non-members. You can join the Foundation at https://resilienceinsoftware.org/signup   Zuul is what Volvo uses for their CI, and it’s part of the OpenInfra Foundation , it’s rad. You can find Andrew on LinkedIn here .

Michael Wettick’s Lund thesis is great , and Laura Maguire’s paper on the Costs of Coordination that is a shortened version of her dissertation is worth a read! Clint’s SRECon talk that he mentioned a couple times: https://www.youtube.com/watch?v=k4UaDDkLOhw Lorin wrote a great article on incidents and improvisation: https://surfingcomplexity.blog/2023/06/11/when-theres-no-plan-for-this-scenario-y...

The O’Reilly book on Chaos Engineering by Casey and Nora Jones is here: https://www.oreilly.com/library/view/chaos-engineering/9781492043850/ Some of the Netflix posts introducing Chaos Monkey and Simian Army are here and here . You can see Lorin Hochstein talking about Chaos Engineering at Netflix here . The Void is an awesome collection of information on incidents throughout tech and you can fin...

Clint wrote the Socio-Technical Reality Engineer as a blog post it’s a good read. The Burnout book by the Nagoski sisters is A+++ reading. Those Found Responsible Have Been Sacked is by the late, great, Dr. Richard Cook and Chris Nemeth The Perverse Incentives of Reliability by Katie Wilde from Snyk at this year’s SRECon was just an incredible talk. Colette mentioned the Beaumaiden report from DMA...

Ben (Goodheart), Dave (Provan) and Ron (Gantt) have the very awesome podcast Punk Rock Safety (punkrocksafety.com) - you can get your own punk rock safety merch at punkrocksafetymerch.com Charles Perrow wrote Normal Accidents and talks about safety and power in his essay (book, really), Complex Organizations . Drew Rae lit the stage on fire about safety work as soothing rather than actually improv...

No video for this one because it didn’t really end up working. We had some awesome people with us for this show: Eric Dobbs Will Gallego Juan Carlos Ramirez Martin Smith Dr Richard Cook’s talk on The Marvelous Resilience of Bone (one of our absolute favorites) You can see the schedule for the SRECon 2025 Americas conference here The keynotes from the day we recorded were Dr David Woods and Katie W...