February 9, 2016

Why Most A/B Tests Give You Bullshit Results

Most new testers walk into A/B testing thinking it’ll be quick and easy to get results. After running a handful of simple tests, they think they’ll find the right color for this button or the right tweak to that subject line, and conversions will, poof, increase by 38% like magic. A study by AppSumo reveals that only 1 out of 8 tests produce results. Kaiser Fung estimates that 80 to 90 percent of the A/B tests he’s run yield statistically insignificant results.

 Valuable Lessons for A/B testing

Topics: User Experience

Recent

Popular