If you’ve ever shopped for a new vehicle, you’ve made a mental list of the features that most interest you. Is it red? A two-door? And does it have the 19″ alloy wheels?

What’s great about these lists is that they make the buying experience simple. If the car isn’t red, look no further. The challenge, though, is when safety, reliability, and performance are on your list. To evaluate these, you need to be behind the wheel, under the hood, and in between the seats. Unfortunately, when a car doesn’t measure up, it’s not always obvious at first glance.

And so it is with testing solutions. KPIs, including conversion rate and AOV, rank only among the more visible ways of measuring a tool’s impact on your business. The others? Get ready to look under the hood at ease of initial setup, ease of ongoing use, and how risky use of the tool is in the first place.

But how can a tool be “risky”? Like a car without airbags, some tools simply aren’t built with the proper fail-safes to protect companies against the unexpected, including:

  • Sharp increases in traffic
  • Sudden website changes that affect ongoing campaigns
  • JavaScript that is slow to load (or that doesn’t load at all)
When any of these risks becomes reality, the impact on the shopping experience ranges from the “annoying” to the “catastrophic.” And the worst case scenario: your website goes down, transactions cease, customers leave—and they don’t come back. But it doesn’t have to be that way.

From the bottom up, the best website optimization tools are built to eliminate these risks. Here are “3 Under-the-Hood Capabilities to Demand in a Testing Solution.”

1. Cloud-based, Client-side Architecture: Sharp and sudden increases in traffic can overload a server, quickly taking a website down. Your testing solution shouldn’t be cause for concern. Cloud-based, client-side tools operate independently of your server, hosting center, and ecommerce platform. They utilize vast, redundant, globally-distributed server networks that offer near-infinite scalability, and perform all necessary tasks (data collection, decisioning, content rendering, etc.) via a single round-trip. Some testing solutions require multiple server calls for each individual test element. Others reroute your traffic through a proxy, creating a single point of potential failure should that proxy ever fail. But the world’s biggest, highest-traffic websites employ cloud-based architectures. Your testing solution should too.

2. “Look Before You Leap” Campaign Audit: Quality Assurance should have redundancies, involving both the client success professionals you work with on a regular basis, as well as automated controls that detect website changes that might conflict with active campaigns. When your testing solution can auto-detect site changes that are incompatible with a campaign, it’s able to disable that campaign in real time and alert your services team. Without this ability, the shopping experience can “break,” yet go unnoticed for an extended period of time.

3. Ability to Safely Place the Tool’s JavaScript at the Top of the Page: JavaScript tags from third-party vendors (such as your analytics platform) are commonly placed at the bottom of the page source code. And historically, this has been fine! In general, because code executes in the order it appears on the page, it was (and continues to be) perfectly acceptable for tags that record the result of customer experiences to be at the bottom of the page. But your testing solution is different. The first task of your testing solution is not to record the results of a customer experience, but to serve the experience itself.

As a result, to avoid the customer seeing the web page “change” when Test content has to replace Control content that’s already loaded, your testing solution’s JavaScript should be placed as close to the top of the page as possible (ideally in the Header). But you must be able to do this safely. Some JavaScript tags load synchronously. Synchronous JavaScript tags form a queue, in which no tag can start executing until the one before has finished. This can delay your campaign content from loading. Even worse, if the synchronous tag fails to load properly, none of the synchronous tags beneath it will load either. In effect, this brings your website down, and if this were to happen on Cyber Monday, how much money would your company lose—every minute?

Look for a testing solution whose tag loads asynchronously. In non-technical terms, this means it loads independently of all other JavaScript on the page. No other tag will delay your campaign content from loading. And just as important, the asynchronous tag won’t delay anything else. As a result, it can safely be placed at the top of the page.

When it comes to selecting a testing solution, many vendors speak of potential return on investment. To be sure, campaigns that prove their worth through incremental annual revenue and high statistical significance are essential ingredients. Still, another dimension of ROI includes the impact on revenue when testing solutions don’t always perform as they should. When these risks materialize, your site can lose money—and lots of it.

A great testing solution, then, emphasizes both sides of the coin. Business-relevant features on “Heads” and proper fail-safes on “Tails.” Make it your New Year’s Resolution to demand this in 2012.