A/B testing most commonly FAILS because the test itself has unclear GOALS, so you've got to know what you're testing. USE A/B testing to test a theory, for example -- would adding a picture to this LANDING page INCREASE conversions? Are people more likely to click a red button or a blue button? What if I change the headline to stress the time-limit of the offer? These are all changes that can be easily quantified. People run into trouble with A/B testing when their theories are too vague, like testing two entirely different designs with multiple variants. While it can be done, unless there is a clear landslide winner, testing different designs can lead to softer conclusions and an uncertainty about what actually caused the increase in conversions.

A/B testing most commonly fails because the test itself has unclear goals, so you've got to know what you're testing. Use A/B testing to test a theory, for example -- would adding a picture to this landing page increase conversions? Are people more likely to click a red button or a blue button? What if I change the headline to stress the time-limit of the offer? These are all changes that can be easily quantified. People run into trouble with A/B testing when their theories are too vague, like testing two entirely different designs with multiple variants. While it can be done, unless there is a clear landslide winner, testing different designs can lead to softer conclusions and an uncertainty about what actually caused the increase in conversions.