A/B Testing Vs. Multivariate Testing

Writing by Nick Stamoulis Tweet

Testing your landing pages is one of the most important parts about being an effective online marketer. You should be testing constantly. But there are different types of tests. You can run A/B tests or multivariate tests and which one you use on your site depends on a number of factors.

A/B testing is when you have two pages, or two elements on the same page, that you want to test side by side. For instance, let’s say you have a landing page that currently converts at 5.5%, but you think you can increase that by changing your headline. So you rewrite the headline and test the new headline against the old headline. This is an A/B test. After a predefined number of page views for your new web page, you’ll have a clear picture on which headline is better for conversions. If your old headline wins then you do nothing. If your new headline wins then you make that page live – nothing on the page changes other than your headline.

A multivariate test is a bit more involved. In this case you are testing multiple elements at the same time, hence the name. By testing multiple variables simultaneously, you can arrive at a conclusion much sooner. Maybe you want to know which headline of three and which call to action of two are most effective. You can determine that through multivariate testing.

Another way to conduct multivariate testing is by running a test on a combination of variables. Suppose, for instance, you want to know which mix of headlines and calls to action will be most effective. You can test that with multivariate testing as well.

Which type of test you run for your landing pages depends on your goals and the individual web pages you are testing. There are times when you’ll use the A/B test and then other times when the multivariate test is more appropriate. But one thing is clear: You’ll want to develop your testing of landing pages strategy early on and always work toward improvement.