30 Multivariate & A/B Split Testing Tools, Tutorials & Resources
Getting targeted traffic for a website is just the first step in a thorough SEO process. What comes next? Landing page optimization or conversion (rate) optimization/CRO follows. After all you want the visitors you get to stay on your site and perform the desired action (like ordering or subscribing).
One of the foremost ways to optimize landing pages for more conversions is A/B split testing
sometimes also referred to as simply AB testing. The concept is pretty simple. In the most basic set up you show version A of a page to half of your visitors while in the other half will see version B.
Of course in reality it’s a bit more complex.
You will have more than two versions in many cases and you won’t split 50/50 but will choose a smaller portion of your overall traffic to test. Also you will probably only subject new visitors to your tests in some cases in order not to scare away your repeat visitors by frequent changes.
When you prepare more than two versions of your content (as in A, B, C etc.) it’s not necessarily called multivariate testing then. Multivariate testing rather refers to tests with multiple variables you check.
The most known tool for A/B split testing is Google Website Optimizer. Like with many free tools and similarly to Google Analytics it’s not the easiest to use tool though.
There are more user friendly tools out there by now. Over the course of the last year A/B split & multivariate testing has been booming. I discover new tools, tutorials and other resources every other day. Nonetheless it’s by now a best practice that has been around for years. So this is not about early adopting a methodology. AB testing is common sense by now on the Web.
Introductions to A/B Split Testing
- What is AB Split Testing?
- A/B Split Testing
- How A/B Split Testing Works
- A/B Split Testing Crash Course
- An Introduction to Website Split Testing
- Fundamentals of A/B and Multivariate Testing
Examples and Case Studies
- A/B Testing for the Mathematically Disinclined
- A simple multivariate testing strategy that produced a 367% increase in downloads
- A/B test case study: how two magical words increased conversion rate by 28%
- Twitter A/B testing sign-up button (at least 5 versions)
- ABtests.com – Share A/B Testing Results.
Resources
- Gathering Variables for A/B Split Testing
- How to get proven ideas for your next A/B split test?
- Significance of Statistically Significant Results in A/B Testing
Tools
- Website Optimizer
- Split Test (A/B Test) Calculator
- A/B Theme Testing Plugin for WordPress
- Visual Website Optimizer – A/B Split Test and Multivariate Testing Software
- A/Bingo: RoR Split Testing
- LiveBall Platform
- genetify – Beyond A/B testing
- Unbounce Create, Publish & A/B Test Without I.T.
- Vertster – Multivariate Testing and A/B Split Test Software
- Performable – Convert More Visitors into Customers
- Sitespect – A/B split Split Testing
Guides & Tutorials
- The Ultimate Guide To A/B Testing
- A/B testing to increase your conversions the Eric Ries way using Mixpanel
- 3 very different ways to scale your landing pages
- 55 Google Website Optimizer Tips and Tricks
- Google Website Optimizer – 108 Free Tips
The biggest problem with A/B and multivariate testing is the time and resources needed. Even a simple test for small website entails three different departments be it, SEO, analytics, web design.
How do you perform your A/B tests? I am still undecided which tools best suit my requirements.
July 2nd, 2010 at 1:28 pm
Hi Tad,
Thanks for putting up this excellent list of A/B testing resources! Clearly, there is a plethora of educational material available and lack of awareness doesn’t seem to a problem. According to my experience, A/B testing is something many online businesses want to do, but the plain-old inertia prevents them from deep-diving into it. They still have a fear that it isn’t they can do easily.
A/B and multivariate testing technology isn’t the bottleneck. For example, my startup Visual Website Optimizer http://visualwebsiteoptimizer.com/ is laser-focused on making testing as simple as possible. Contrary to your point, it tries to hand over control of testing initiatives to the marketer, eliminating IT and other departments from the loop. In fact, I’d say it takes “effort” and “commitment” out of the question.
The major stumbling block in A/B testing adoption is lack of motivation or it-isn’t-for-me attitude. I think the issue of not doing enough A/B testing is entirely behavioral, not related to methodology or tools involved. What do you think?
July 2nd, 2010 at 3:14 pm
That’s quite a list. The examples and case studies are probably the best resource of all.
July 3rd, 2010 at 6:42 pm
Really love the Visual Website Optimizer. It’s a great tool for advertisers, especially to become independent from IT!
July 5th, 2010 at 12:56 pm
Great job with the list covering all aspect of conversion rate optimisation from strategy to the practical implementation.
July 6th, 2010 at 5:23 pm
[...] 30 Multivariate & A/B Split Testing Tools, Tutorials & Resources – SE Optimize [...]
July 7th, 2010 at 1:31 pm
Great posting, good to see more and more resources avaialble to marketers on this subjec that not so long ago was quite tricky to fully research.
You missed Maxymiser from your list of tools, hope you won’t mind me posting here to bring it to your readers attention. We provide a full service testing solution from offices in the USA, UK and Germany.
July 8th, 2010 at 9:59 am
Oh, great! I always like opportunities to learn more about SEO, especially now when I’m fighting to improve it on my blogs, thanks.
July 12th, 2010 at 12:08 pm
Hey Paras, thanks for the feedabck. You’re tool had been on th elist already!
David: Yeah, the examples are insightful, even the simple Twitter example. That’s why I inclueded not only the tools.
Dan: We have another, even better CRO list linked in the above post at the top:
http://www.seoptimise.com/blog/2010/05/46-croconversion-rate-optimization-resources-for-web-design-seo-social-media-experts.html
Alasdair: Thanks for the addition but I can’t find much info about the actual tool on your site. Features, screen shots, pricing? Sounds more like a “solution” aka service.
July 12th, 2010 at 12:45 pm
Hi Tad,
Indeed, we don’t publish pricing but you can find more about the tool in the datasheet linked to at the base of the page below. Our main differentiator is the One Touch deployment method which is one tag one time meaning no re-tagging or tag changes between tests:
http://www.maxymiser.com/solutions/one-touch-methodology
July 26th, 2010 at 6:58 am
Great job! I like the most is case studies and tutorial which no one provide and if provides there are very rare case. Providing tools is not great work but providing all the useful stuff related to that is the most important job of all.
August 5th, 2010 at 3:38 am
[...] of the foremost ways of optimizing your web design is via A/B testing(sometimes referred to as split testing). An A/B test examines the effectiveness of one landing page [...]
August 5th, 2010 at 9:31 am
[...] 30 Multivariate & A/B Split Testing Tools, Tutorials & Resoures [...]
August 9th, 2010 at 10:35 pm
Really good list, it went straight to my delicious.com profile! :)
I’ve released a tool for checking the statistical significance of a split test’s results. It’s free and I invite everyone to give it a try.
http://newinternetorder.com/how-to-check-the-significance-of-an-ab-split-test/
August 10th, 2010 at 7:02 am
[...] usability and conversions.One of the foremost ways of optimizing your web design is via A/B testing (sometimes referred to as split testing). An A/B test examines the effectiveness of one landing [...]
August 10th, 2010 at 8:02 am
[...] of the foremost ways of optimizing your web design is via A/B testing (sometimes referred to as split testing). An A/B test examines the effectiveness of one landing [...]
August 10th, 2010 at 1:49 pm
[...] of the foremost ways of optimizing your web design is via A/B testing (sometimes referred to as split testing). An A/B test examines the effectiveness of one landing [...]
August 17th, 2010 at 9:18 pm
[...] 30 Multivariate & A/B Split Testing Tools, Tutorials & Resources … – Crossroads by Captain DJ. Getting targeted traffic for a website is just the first step in a thorough SEO process. What comes next? Landing page. [...]
August 18th, 2010 at 10:26 am
[...] of the foremost ways of optimizing your web design is via A/B testing (sometimes referred to as split testing). An A/B test examines the effectiveness of one landing [...]