Establishing and following a governance structure for website optimization is essential to your continued success. In my first blog post on website optimization governance, I covered the foundational steps to build an internal structure for website optimization. In my second blog post, I discussed how you might gather, vet, and prepare testing ideas.

If you’ve done all the work laid out in the first two posts, I have some good news, and some bad news, for you. The good news is that if you’ve done the work to establish a strong governance structure, the work moving forward into live tests will be more straightforward and efficient. After all, without a clear governance structure in place, the tasks of launching, monitoring, and iterating tests almost defies organization after the fact. And the bad news? You’ve finally arrived at the moment where you should be launching tests and everything is about to get harder and more complex.

Launch Tests

Now that you have concepts vetted and developed for launch, it’s time to get them live. But there are a few considerations to think about before you hit ‘activate’ or merge in the test-code: Make sure you have a pre-defined plan for where the test could generally move—positive, neutral, or negative to your business or KPIs (Key Performance Indicator). More on that later.

Define a KPI, or set of KPIs, that will be your focus for the test. Also consider if there are any other custom KPIs, such as click-through or non-revenue conversion, that you plan to track for the test. Usually these custom KPIs will need to be defined and set up prior to the launch. Also, make sure you inform stakeholders that the test is about to launch. Stakeholders might include relevant functional areas, submitter of the idea, or developers/PMs involved in the assembly of the test.

Monitor & Analyze Results

As tests are active and start to gather data, be aware that significance levels can evolve. For that reason, it makes a lot of sense to have some standard procedure for calling a test a winner, a dud, or an outright loser. We all hope that our ideas are dripping with genius and potential, but your customers are always the best judge about what works and what doesn’t.

At the minimum, you should strive to have operating procedures for when stakeholders should focus on results of an active test. A two-week duration is a rough starting point to account for weekly sales and traffic cycles, though the answer for your website can vary widely by site traffic, test traffic, test impact, and segmentation. If a test is high-visibility or a key initiative, you should of course keep close tabs on the results as they evolve.

Iterate on Winners, Duds, and Losers

When you launched your tests, you should already have had a sense for what will typically happen in each test-result outcome: positive, negative, neutral. Now that a test has been live long enough to stabilize and present statistically significant results, it’s time to iterate.

Here are some things to think about:

For positive tests:

• Are there ways to further improve upon a test? Don’t fall into the “trophy syndrome” where you put a winning test on the shelf and let it gather dust.

• Could a different creative work even better than the current creative?

• Might the concept work even better for certain segments?

• Could the concept work even better on different on-page locations (same as current)?

• Would the concept work even better on different pages, further up or down the site experience or conversion funnel?

For neutral tests:

• Same as above, but ask yourself if there are other tests in the pipeline that might be better suited for attention.

For negative tests:

• Same as above, but there will more likely be significant changes to the test concept than in the first two cases above.

• Decide on your comfort level for risk. For most testing teams, it’s fine to have this loss threshold quite low. If a test comes in with stable and negative impact, there’s a low likelihood that allowing the test to continue to run will move all the way to a positive result.

These are only guidelines and there are sure to be exceptions. Make sure you approach this as a process, not a science. Website optimization and personalization is a challenging, interesting, and dynamic endeavor, but approached in a clear way, the art and the science online optimization can really accelerate your goals and your organizations understanding of your customers.

Reach out to me at wharries[at]monetate.com if you have any questions. And remember: website optimization is certainly a journey with many twists and turns, but those are also the hallmarks of a journey well worth taking.

Will Harries is a Strategic Services Director at Monetate. He is an ecommerce specialist with substantial experience in end-to-end retail operations, consumer market research, strategy development, and P&L improvement.

Quality Process Sticky Paper image courtesy of Shutterstock.