Experiment Nation: The Podcast

What do you do when an experiment tanks conversion rate by 8% in less than a week? In this episode, Gerda and Rommil break down real CRO and experimentation scenarios submitted by the community, including failed tests, executive pressure, guardrail metrics, knowledge management, developer friction, and why experimentation teams always seem to end up back in spreadsheets. Check out: http://katsed.c...

Most teams say they “do research”… but very few actually do it well. In this episode, we break down what real customer research looks like in CRO and growth. From analyzing 200+ survey responses manually, to why AI still struggles with qualitative data, to the politics that slow everything down. We also get into:Why AI fails at nuanced research (and what it misses)How to actually analyze open-ende...

In this episode, we dive into the realities of CRO (Conversion Rate Optimization) today — from AI anxiety to experimentation culture and how teams should actually be structured. We start with a surprisingly philosophical take on photography and overthinking, then quickly shift into what’s really happening in the CRO and AI space right now. We discuss: * Why AI might be making everything look the s...

In this episode, we answer real, unfiltered questions from the CRO and experimentation community. We dig into how far AI can realistically help with experimentation, why revenue metrics like ARPU are a nightmare for MDE calculations, how to handle sketchy interview “audits,” and why ROI conversations never seem to end. No hype. No magic bullets. Just honest perspectives from people actually doing...

A real conversation about why A/B testing doesn’t magically fix broken businesses. We break down the biggest CRO myths, why copying case studies fails, how experimentation tools really get chosen, and what actually causes friction between CRO, UX, and research teams. For CROs, experimenters, product, growth, UX, and analytics leaders.1:38 The biggest myth in A/B testing3:00 The one-person CRO team...

In this episode, we go off-script and cover everything from CRO strategy to existential questions. We talk through:How to recover after being shaken at workWhy long-form content still matters in experimentationCRO differences between B2B, SaaS, and ecommerceWhy website redesigns without A/B testing backfireHow beginners can gain real CRO experienceWhat it actually takes to get people to listen to...

Koalatative's Gerda Thomas and Experiment Nation's Rommil Santiago answer questions from the CRO community. Chapters3:35 – Is there room in CRO for AI skeptics anymore?4:58 – How can I feel like I'm using my CRO skills for good?11:29 – AI is boring and speeding up the decline of civilization. Agree or disagree?15:22 – How do I tell my micromanaging manager to shut up?18:43 – Why are you the way th...

Koalatative's Gerda Thomas and Experiment Nation's Rommil Santiago answer CRO questions from the community. Chapters:1:51 – What's the one thing people on internet still refuse to automate or use AI for that you think they should?”5:07 – How do you measure the impact of your CRO program over time?10:37 – What to do when leadership has a win at all cost mentality, choosing short-term wins over long...

Koalatative's Gerda Thomas and Experiment Nation's Rommil Santiago answer your CRO questionsPart 1 of 3Chapters:8:05 – What is the right balance between big tests versus small and medium tests?12:47 – How many iterations per test will you run knowing the first test was negative but still see some relevant data toward the hypothesis?14:32 – In small traffic sites, is it best to run one A/B test or...

In this episode of Experiment Nation, host Jason Gossit sits down with Ameliano Blanca, a senior growth and product leader with 10+ years of experience driving revenue through data-driven experimentation. They dive deep into:How to build experimentation programs from the ground upThe balance between qualitative and quantitative insightsExperimentation in startups vs. mature organizationsCRO framew...

In this episode of Experiment Nation, host Jason sits down with Namata Agarwal, a seasoned product designer with over 17 years of experience building digital products, websites, and apps. Together, they explore:- Ethical testing: how to balance user needs with business goals.- Leaks vs. puddles framework: a practical way to identify urgent vs. non-urgent product problems.- The role of design in CR...

In this episode of the Experiment Nation Podcast, host Ward Vanestra sits down with Ryan Lux, one of the first team members at Eppo and now Experimentation Evangelist at Datadog. They dive into:What Eppo is and how it helps companies experiment at scaleWhy data warehouses are critical for trustworthy resultsThe recent Datadog acquisition of Eppo and what it means for the futureAdvanced methods lik...

In this episode of the Experiment Nation Podcast, host Richard Joe welcomes Manuel Da Costa, founder of Effective Experiments and now Efestra, to unpack a critical issue in the experimentation industry: the trust gap between practitioners and business decision-makers.🎯 Manuel shares how running more experiments isn’t the goal—making better decisions is. He introduces the Learning Loop, explains w...

In this episode, we dive into Speero’s shift from CRO to Experiment-Led Growth, how they structure their testing systems, where to start with a new client, and what makes a good growth bet. Plus, we discuss the power of insights, the rise of EXL meetups and the future of Speero. Chapters----00:00 - Introduction 00:18 - Meet the Hosts & Guest 01:11 - From CRO to Growth Experimentation 02:55 - B...

This episode features Achille Vaulot, a CRO expert, discussing the evolution of conversion rate optimization (CRO) and the steps to building a mature program. He highlights the shift from basic A/B testing to a data-driven, strategic approach encompassing research, thorough analysis (including post-test analysis of both primary and secondary KPIs), and effective client communication. Achille empha...

Ekaterina (Shpadareva) Gamsriegler, a seasoned growth and marketing expert, shared her insights on mobile app experimentation. Here are 5 key takeaways: Mobile experimentation is more complex: Release cycles are longer, and dependencies on app versions can slow down learning. Think beyond the app: Experimentation should encompass ads, store listing pages, and the app itself. Borrow insights from o...

Tracy Laranjo shares learnings from years of growing her career with Suchita Srinivasan, here are some key takeaways: 1. CRO vs Experimentation: CRO focuses on driving performance and results, while experimentation centers around decision-making and validating ideas. 2. Agency vs In-House: Tracy’s journey highlights the benefits of agency experience for rapid skill development and exposure to dive...

This episode of Experiment Nation features an interview with Dr. Else van der Berg, a product management consultant for B2B SaaS startups and scale-ups. Dr. Else van der Berg discusses her process for helping companies become more outcome-driven, focusing on understanding the ideal customer profile (ICP) and its associated value. She also shares insights on common challenges faced during the testi...

We recently had the pleasure of chatting with Rishi, a Shopify product expert. Here are 5 key takeaways for optimizing your product pages: 1) Product pages are the Grand Central Station of conversions. They’re where buyers make the decision to buy or leave. 2) Pop-ups aren’t as effective as you think. Focus on valuable content, not annoying interruptions. 3) Content is king (but don’t be afraid to...

Black Friday is coming, and the pressure is on. But don’t worry, Ilan Hurwitz has some expert advice to help you navigate the chaos. Here are five key takeaways from his chat with Tracy Laranjo: 1) Start now! Don’t wait until the last minute to get buy-in and start testing. 2) Speak the language of profit. Showcase the ROI of CRO to your stakeholders, focusing on increasing conversions and profits...

Experimentation is a journey, not a destination. Don’t expect to build a culture of experimentation overnight. Cross-functional teams are critical. Breaking down silos and bringing together engineers, product managers, and designers will create a unified vision and drive faster iteration. Leadership buy-in is essential. A top-down approach to embedding experimentation ensures everyone is on board...

In this podcast episode of Experiment Nation, hosted by Charlotte Bomford, guest Iqbal Ali, an experienced web designer, researcher, and comic book writer, discusses the concept of solution bias and the importance of identifying underlying problems before diving into solutions. Ali emphasizes using AI to help understand problem landscapes, avoiding hastily jumping to solutions, and improving exper...

Ruben de Boer, a lead consultant at Online Dialogue and the founder of Conversion Ideas, shares his approach to conversion rate optimization (CRO) and experimentation. With 15 years of experience, Ruben emphasizes the importance of data-driven decision-making and understanding human behavior. He highlights the shift from focusing solely on digital products to also optimizing organizational process...

Tymur Donets, a seasoned Growth PM and experimentation consultant, shared some powerful insights during his Experiment Nation talk. Here are five key takeaways: Growth PMs are holistic: Don’t be siloed! Growth PMs work across the entire customer journey, ensuring marketing and product teams are aligned. Commercial awareness is key: Both Growth PMs and CROs must understand the business impact of th...

Kelly Wortham dropped some serious knowledge bombs in our recent Experiment Nation chat. Here’s what stood out: - CRO isn’t just for marketers: Lawyers, project managers, even developers can thrive in this field. It’s all about a love of learning and adaptability. - Don’t underestimate the power of mentorship: Kelly has THREE mentors, each focusing on different aspects of her career. Find your own...