What's Multivariate Testing, And How Does It Compare To A/b Testing?

1.	What's Multivariate Testing, And How Does It Compare To A/b Testing?
Answer» A/B testing is typically used for redesigns to TEST out the effectiveness of a single design direction or theory against a goal (LIKE driving conversions). Multivariate testing tends to be used for smaller changes over a longer period of time. It will take a number of elements of your site and test out all possible combinations of these elements TOGETHER for ongoing optimization. In a post in January, my colleague Corey Eridon explained the differences between when you'd use one test over the other in detail, saying. A/B testing is a great testing method if you need meaningful RESULTS fast. Because the changes from page to page are so stark, it will be easier to tell which page is most effective. It is also the right method to choose if you don't have a ton of traffic to your site. Because of the multiple variables being tested in a multivariate test, you'll need a highly trafficked site to get meaningful results with MVT. If you do have enough site traffic to PULL off a successful multivariate test (though you can still use A/B testing if you're testing brand new designs and layouts!) a great time to use the testing method is when you want to make subtle changes to a page and understand how certain elements interact with one another to incrementally improve on an existing design.

Answer»

A/B testing is typically used for redesigns to TEST out the effectiveness of a single design direction or theory against a goal (LIKE driving conversions). Multivariate testing tends to be used for smaller changes over a longer period of time. It will take a number of elements of your site and test out all possible combinations of these elements TOGETHER for ongoing optimization. In a post in January, my colleague Corey Eridon explained the differences between when you'd use one test over the other in detail, saying.
A/B testing is a great testing method if you need meaningful RESULTS fast. Because the changes from page to page are so stark, it will be easier to tell which page is most effective. It is also the right method to choose if you don't have a ton of traffic to your site. Because of the multiple variables being tested in a multivariate test, you'll need a highly trafficked site to get meaningful results with MVT.
If you do have enough site traffic to PULL off a successful multivariate test (though you can still use A/B testing if you're testing brand new designs and layouts!) a great time to use the testing method is when you want to make subtle changes to a page and understand how certain elements interact with one another to incrementally improve on an existing design.

Discussion