When you don’t know a whole lot about your customers, the point of testing is obvious. You have to guess what’s going to work for them. Sometimes you guess right, sometimes you guess wrong. Very often, the outcome is no lift at all, and that’s usually because what you thought was important didn’t matter to them.

When you know more about your customers―including past browsing history, in-session behavior, third-party data, and more―you don’t need to guess as much as you did before. You no longer need to test as much because what you know about your customers makes the outcomes much more predictable.

Thanks to advances in post-hoc segmentation, cohort analysis, and automated segment discovery, you are able to know your customers’ interests and preferences. This insight allows you to skip the testing process, go straight to identifying over-performing and underperforming customer segments, and then correlate those segments with the experiences that drive the observed outcomes.

The result is something very specific: Take all of the attributes about a customer, and understand what’s important (and not important) to them. You also can know from the first touchpoint what experience you need to show them.

The problem with testing―in particular, testing that’s not finely targeted―is that performance just converges toward a mean. So it may be the case that there’s no lift at all because one customer prefers a red button and another prefers a blue button, thus their behaviors cancel each other out.

The advances in analytics help you move beyond traditional testing, which requires a marketer to know whom they want to test―and what they want to show―upfront. You’re basically asking the question: Who is my customer? With big data and better analytics, you already know who your customer is because people intrinsically are different from each other.

Up until now, you might typically have run a test that suggests that a red button works slightly better than a blue one―so you would promote the red button to 100% of your visitors and subsequently turn off a portion of your customers. You have so much data about your customers now that you know, before you do any testing, that customers with certain attributes will prefer a red button, while others will prefer a blue one.

This represents a big leap for marketers. For example, suppose you know…

• Visitors in Missouri
• Who are in the 85th percentile of median family income or above
• Who are likely to have master’s degrees
• And who live in cities where it’s currently raining

…perform well when you showcase “Top Sellers” on your home page. You no longer need to test to discover this segment because you’re already sitting on a remarkable amount of data that tells you who these customers are and what experiences they like.

So when do you need to test? You need to test when you don’t have enough customer data to facilitate automated segment discovery and cohort analysis. In other words, testing will remain something that small companies do, simply because it’s the cheapest (and still a very practical) way to learn more about their customers.

For larger companies that want to streamline and scale their personalization programs, automated segment discovery is paving the path to a new set of testing best practices that I will discuss in my next post.

Arrow Hits Bullseye and Many People Speak images courtesy of Shutterstock.