Conversion Rate Optimization with Fexingo: A/B Testing, Landing Pages, and CRO Strategy

Episode 57 of Conversion Rate Optimization with Fexingo digs into a phenomenon that catches many CRO teams off guard: the reversal effect, where a winning variant in an A/B test starts losing after the test is stopped. Lucas and Luna walk through a real case from a mid-market SaaS company that saw a 12% lift during the test, only to watch it vanish within three weeks post-launch. They explain how...

Episode 56 of Conversion Rate Optimization with Fexingo digs into a common but dangerous mistake: treating all users the same in your A-B tests. Lucas and Luna explain why averaging results across different audience segments — like new vs. returning visitors, mobile vs. desktop, or weekday vs. weekend traffic — can hide a winning variant that actually hurts a key subgroup. Using a real example fro...

Episode 55 of Conversion Rate Optimization with Fexingo tackles a frustrating A/B testing paradox: what happens when both of your new variants outperform the control, but one doesn't statistically beat the other? Lucas and Luna walk through a real example from a mid-market SaaS company that ran a three-way test on their pricing page. The control had a 3.2% conversion rate, Variant A hit 4.1%, and...

Lucas and Luna tackle the uncommon but maddening A/B testing failure mode where multiple test variants each beat the control, but no single variant is statistically superior to the others. Using a real example from a SaaS onboarding flow, they walk through how this 'flat winner' pattern emerges, why it often points to a fundamentally weak hypothesis, and what to do next — including the one retest...

Episode 53 of Conversion Rate Optimization with Fexingo dives into a common but overlooked A/B testing pitfall: pre-registration bias. Lucas and Luna explain why logging your hypotheses, sample sizes, and success metrics before running a test can prevent p-hacking and false positives. They walk through a real example from a B2B SaaS company that halved its false-positive rate by adopting a public...

In this episode of Conversion Rate Optimization with Fexingo, Lucas and Luna dive into the hidden pitfall of post-hoc analysis: why even a perfectly executed A-B test can lead you astray if you peek at the results before the sample size is reached. Using the real-world example of a travel-booking site that accidentally cut revenue by 8 percent after 'winning' a test early, they explain what a pre-...

Episode 51 of Conversion Rate Optimization with Fexingo dives into a cautionary tale: how a SaaS company lost 40% of its trial-to-paid conversion by testing button color—without controlling for user intent. Lucas and Luna unpack the difference between surface-level A/B tests and meaningful behavioral experiments, explaining why the same button color can double or halve conversion depending on whet...

Lucas and Luna dive into one of the most overlooked errors in A-B testing: miscalculating sample size. Using a real-world case from a mid-market SaaS company, they show how a seemingly valid test with 10,000 visitors per variant actually needed 50,000 to detect a realistic 2 percent lift. They explain the math behind statistical power, why most online calculators give misleading defaults, and how...

As peak shopping season approaches, Lucas and Luna dive into a common but costly CRO mistake: treating holiday A-B tests like any other experiment. They unpack why seasonal spikes in traffic and buyer intent change the sample size math, using a real example from a mid-size fashion retailer whose Black Friday test broke a winner that flopped in January. Listeners learn about the 'seasonal validity...

Episode 48 of Conversion Rate Optimization with Fexingo dives into the hidden cost of page speed. Lucas and Luna unpack a real case from mid-2025: a mid-market fashion retailer that lost an estimated 11 percent of online revenue — roughly $4.7 million annually — because its product detail pages loaded one full second slower than the industry benchmark of 2.5 seconds. The duo walks through how the...

Episode 47 of Conversion Rate Optimization with Fexingo dives into the novelty effect—why users behave differently with a new design simply because it's new, not because it's better. Lucas and Luna use a real-world example from a SaaS company that saw a 25% lift in the first week, only to see it vanish by week three. They explain how to detect novelty effects with time-segmented analysis and holdo...

Episode 46 of Conversion Rate Optimization with Fexingo dives into the debate between Bayesian and frequentist statistical methods in A-B testing. Lucas and Luna unpack a real case from a mid-market SaaS company that ran a 10-variant test: one statistically significant winner under frequentist logic, but Bayesian analysis suggested all variants were essentially tied. They explain what prior probab...

Lucas and Luna dig into a surprising A/B test where changing the number of checkout form fields from eight to six actually decreased conversions. They explore why reducing friction isn't always the answer, how user intent reshapes form design, and what a notorious 2009 Expedia test reveals about the difference between perceived effort and real effort. Along the way, they discuss a 2.4 percent drop...

Episode 44 of Conversion Rate Optimization with Fexingo. Lucas and Luna explore the counterintuitive finding that deliberately slowing down a landing page can increase conversion rates. Using a real-world case from a mid-market SaaS company that A/B tested a 300-millisecond artificial delay, they unpack the psychology of perceived effort, the distinction between performance and patience, and when...

Episode 43 of Conversion Rate Optimization with Fexingo dives into one of the most common and costly mistakes in A-B testing: peeking at results before the experiment ends. Lucas explains the statistical problem of 'peeking bias' — how repeatedly checking significance inflates false positives — and illustrates it with a concrete example from a real e-commerce checkout flow. Luna pushes back on the...

Lucas and Luna tackle a subtle but dangerous threat to A-B test validity: the novelty effect. Listeners learn how early spikes in conversion often mask users' initial curiosity rather than true preference change. The hosts walk through a real case from a mid-market SaaS company that saw a 6% lift on the first day of a test only to watch it evaporate by day seven. They explain how to spot the patte...

Lucas and Luna explain 'peeking,' the practice of checking A-B test results early, which inflates false positive rates. They break down why stopping a test the moment it hits 95% confidence is a statistical trap, using examples from Optimizely and booking sites. The episode explores the concept of sequential testing and the 'always valid' p-value method, then discusses how persistent peeking led o...

Episode 40 digs into the 'winner's curse' of A-B testing: why a variant that crushes it in an experiment often fails when rolled out to all users. Lucas and Luna unpack the behavioral cause — novelty effects, seasonal confounds, and Simpson's Paradox — using real examples from ecommerce and SaaS. They explain how to spot a false positive before you ship, and why a holdout group is your only safety...

When The Economist redesigned its checkout flow, something strange happened: conversions on the paid subscription page dropped by nearly a third. The culprit wasn't the price, the copy, or the call-to-action button. It was a single text input field — a 'Company Name' box that appeared only on the payment screen. In this episode, Lucas and Luna trace how a seemingly harmless form field added fricti...

Most A-B tests are designed to detect a 10% or 20% lift — but what happens when the true effect is only 3%? In this episode, Lucas and Luna break down the concept of Minimum Detectable Effect (MDE) and why ignoring it leads to false negatives and wasted traffic. They walk through a real example from an e-commerce checkout flow where a properly sized test uncovered a 4% gain that an underpowered te...

Lucas and Luna dig into a subtle but catastrophic mistake in A-B testing: launching the winning variant to 100 percent of users without a holdout group. Using examples from Skyscanner, Etsy, and LinkedIn, they show how novelty effects, seasonal spikes, and metric degradation can make a test winner look real when it's not. They explain the holdout group method—keeping 5 percent of users on the old...

Lucas and Luna dig into the 'button effect' — the subtle, often invisible ways that changing one element in an A/B test (like a button's size, color, or wording) can ripple through user behavior and distort results. They use a real example from a SaaS company that tested a green versus red CTA button: the red button won by 12%, but when they dug deeper, they found it was actually driving fewer hig...

In this episode of CRO with Fexingo, Lucas and Luna dig into the phenomenon of vanishing lift: why an A-B test that shows a clear winner in the experiment often fails to move the needle after the winning variant goes live. Drawing on a case study from Etsy, where a button redesign showed a 5 percent relative lift in the lab but zero impact in production, the hosts explore three root causes: novelt...

Episode 34 of Conversion Rate Optimization with Fexingo tackles a painful, rarely discussed failure mode: the winner that vanishes the moment you deploy it. Lucas and Luna dig into the 'novelty effect' and the 'Hawthorne effect' — two behavioural phenomena that inflate A-B test results during the experiment but evaporate once the change becomes permanent. They walk through a real case study from a...

In this episode of Conversion Rate Optimization with Fexingo, Lucas and Luna explore a case study that often gets overlooked: how a small e-commerce brand improved its checkout conversion rate by 12% simply by changing its font. They break down the psychology behind typeface readability, the specific A/B test design (including sample size and duration), and why the brand's original font—a stylish...