Loss Prevention: The Hidden Benefit of Split Testing

Website optimization testing is becoming an increasingly common practice. It provides quantitative data that proves how making a change to your website will affect order conversion and revenue. I run a lot of tests across all my clients’ sites; my current average is around 400 a year. Collecting data from the past 3 years of testing, only 30% of those experiments were “wins” and positively affected revenue. If you flip that, then 70% of all tests run don’t provide a lift. That seems like a lot of wasted effort.

However, there’s a key component of testing result analysis that’s missing, and a lot of testers fall into this trap. What’s being overlooked are the experiments that proved the current state of the site was better than the proposed change. Basically, if that change had been made without testing, it would have had a negative impact. This is the true hidden benefit of split testing: loss prevention. Using the same time frame as above, an additional 25% of the experiments run were “saves” and prevented a loss in revenue. 30% wins plus 25% saves means around 55% of all experiments run affect revenue.

Diving a little deeper, wins average around a 4.5 – 5% revenue lift per year. Saves, on the other hand, prevent around 6.5 – 7% revenue loss. Saves have a stronger overall effect! For a concrete look at some numbers, let’s examine one client’s testing metrics:

 I will concede that “not losing money” isn’t quite as sexy as “generating more money” but the numbers don’t lie. Loss prevention overall has a more significant impact on revenue. Web optimization testing accomplishes both with a single service.

Are you really, truly certain that the changes you make to your website are beneficial? You might be costing yourself more than you think. I can help you find out. Give me a call at 513-382-3026 or contact me at MBeischel@cohereone.com.