Yes, you guessed it, the topic of this post is testing. More specifically the topic is testing alternative website content and features to produce the best possible traffic conversion rates. I have illustrated some of my points with tests conducted using Monetate’s testing, targeting, and optimization platform. However, the need to test, and the value of testing to the enterprise, is quite independent of Monetate and there are other technologies you can use to conduct tests. Right now I’m more interested in getting you excited about testing in general than about Monetate in particular.

Tafford A/B test at Which Test WonConsider Exhibit A on the left, a classic case of A/B testing. Even when you enlarge this image (by clicking on it) the difference between version A and version B may not be immediately obvious; and when you do spot the difference you might not think it’s very big.

Yet one of these two versions produced 13% more purchases than the other. If you’re in retailing you know that getting 13% more sales is big.

In fact, 13% more conversion of anything online is big, whether it’s a subscription offer, a travel booking, or a request for an insurance quote or product sample. Whatever website interaction you are trying to monetize, it’s a big deal to get 13% more action with a simple change of style, or tone, or content.

Now, can you guess which version of this web page works best? That’s the question posed by a great website that everyone who is into testing should bookmark: Anne Holland’s Which Test Won. Each week Anne features a real world example of an A/B test and asks you to guess which version won. You can then “check your gut” against the actual results. The Tafford Uniform example above was featured on Which Test Won and I will confess my gut got it wrong (the difference between the two versions is that the panel on the right of the page uses text in version A, versus product images in version B). You can read details of the test here.

Testing As Website Marketing Best Practice

A/B test of textThe Bryan Eisenberg digital marketing best-seller “Always Be Testing” firmly established content and feature testing as a enterprise website best practice. In other words, if you plan to use a website to conduct business in a competitive manner, you should be testing key elements of that site to make sure they are doing their job as well as possible, just like Tafford is doing in Exhibit A and the clothing store Free People is doing in Exhibit B (also featured in Which Test Won).

What might surprise you is that the benefits of testing can extend beyond the incremental revenue generated by conversion rate improvements. If you can establish a reliable mechanism for conducting tests quickly and with minimal effort, then testing can have a positive impact on these two aspects of your marketing effort:

  • Creativity
  • Productivity

Testing As Creativity Booster

To claim that A/B testing, an analytical technique, can positively impact creativity might seem counter-intuitive. Surely a technique more “creative” than blind tests should determine the look, feel, and content of the website? To me the answer is clear: Yes and No. I’d say No, you don’t want to let creative judgments go untested or unchallenged. If a soft red “Buy Now” button significantly outperforms a crisp navy blue “Add to Cart” button then aesthetics must give way to fiscal reality. Your goal is to maximize ROI not win design awards (in other words, you are pursuing Conversion Rate Optimization or CRO, not posturing for awards to crow about).

But I’d say Yes, you have to start with creativity. You can’t test something against nothing and you have to start somewhere. For many companies the website starts with the brand, which already embodies, hopefully, a lot of creativity; then there is the list of required content, the principles of user interface design, and all the other elements that go into making the first iteration of a site. This should include all the creativity you can muster. But then you should test, iteratively, and pretty much forever. Why? Because things change, like taste, customer base, and technology (remember, four years ago there wasn’t much call for social network buttons on websites, or video content, or user generated content, and so on).

The way that testing kicks creativity into high gear is by enabling high velocity iteration and a “fail fast” approach to trying new things. People normally frame the benefit of the testing in the positive: Which version produces more purchases? But testing has an equally powerful flip-side, telling you right away if something doesn’t work, which makes it less risky to try something really creative.

Testing As Productivity Booster

The productivity benefits of establishing a system for quick and easy testing of content can be felt at several levels. You may have read David Hayne from Free People talking here about how testing is “sometimes as valuable for what it shoots down as what it confirms.” Or Alex Miller of QVC describing how you can save time and money if you can “test stuff and turn it off if it doesn’t work.”

What we are now seeing in companies that have developed a culture of testing is a streamlining of the marketing process. Time-consuming debates about the potential pay-off from different ideas are now side-stepped with a simple phrase: “Test it and see.”

This can be both liberating and democratizing. After all, there’s no need for people to pull rank or arm wrestle to get their ideas onto the shortlist of candidates for the IT queue if testing is quick, cheap, and easy. Now you can “test it and see” and turn it off if the results are not positive. Or leave it in place to make money if you have a winner.

(If you have a winner, why not share it with Which Test Won and get your marketing chops the recognition they deserve.)

p.s. Want to join a winning team? Monetate is hiring. We’re looking for the brightest and best, engineers and digital marketers. Check out current openings.