Know what to test —and successfully

Now that you’re ready to test, it’s time to decide what to test.

But there are some pitfalls to testing as well. So, in this post, we’ll also take a look at some of the most common pitfalls and also how they can be mitigated.

What should you test?

I have used this handy table for years to talk to clients about testing because it really provided clarity on the kinds of tests that will potentially provide impactful results.

Everyone wants to test subject lines. It’s easy to do and the results come quickly. And getting more people to open and click your emails IS IMPORTANT. However, these types of message tests, once pushed through the entire funnel have much less impact on revenue than say determining what kinds of offers that work best with your target audience.  

Here’s a quick overview of each dimension outlined above:

  • Sources – identify the sources of new contacts and leads that have the most buying power and/or most potential to act
  • Targeting – Test to determine the target accounts that are most likely to be purchasers
  • Offer – Determine which types of offers work best to drive opportunities
  • Channel – Test to find out the optimal channel mix
  • Timing/Frequency – testing to see what days/times of the week work best; also test to determine frequency of communications that work best
  • Messaging – determining which specific messages generate the highest response/conversion
  • Creative – finding the format that generates highest response

Many marketers are surprised to see where they get the most lift from testing. I tell clients to first spend their testing time in the top four dimensions: sources, targeting, offer and channel. Determining the right kind of offer for your prospect during a specific part of the buyer’s journey is going to increase performance a lot more than testing different email templates.

Pitfall 1: Not creating repeatable results

From the list above, you can see that there are so many things that you can test – email vs. telemarketing (channel), white papers vs. webinars (offer), companies with over $500 million in revenue vs. companies with less than $500million (targeting). The list of testing possibilities—and combinations of things you could test—is seemingly endless.

The problem with many tests I have seen is that they don’t give the marketing team results that they can use to make future decisions. For example, subject line testing is very popular. And in most situations, we can get statistically significant results. But what have we learned? In most cases, we have learned that this particular subject line performed better on this particular day with this particular set of contacts. And unless this is an on-going nurture program where we are going to be sending similar kinds of contacts the exact same email in the future, we have not learned anything we can use.

You aren’t  able to tease out which factor made the real difference in the winning outcome. Was it the use of a particular word? Was it a sentiment or overall feeling the subject gave recipients? You just don’t know.

Pitfall 2: Not testing long enough or big enough

If you test a new feature on your site for three days and see lots of activity around it, you may take this as a good sign. And it is a good sign, but it doesn’t definitely mean that your new feature is a home run. Be sure to test for long enough to get a statistically relevant sample size and to cancel out other variables like day-of-the-week.

You can consider your test a success when and only when you have enough data to really see how a new functionality, tool, offer, etc. is impacting business results. So don’t be overly anxious to declare a winner!

Good luck out there!

 When done well, testing can offer insights and data that allow you to make decisions (maybe to change something; maybe to not change something) that will drive measurable business results. Just be intentional about what it is you are testing and ask whether the results of your test will matter (because with statistically insignificant results, you run the risk of making poor decisions).

And, as I’ve mentioned before in my last post, if you can’t track your funnel now, you shouldn’t test. Instead, get your measurement infrastructure shaped up first.

How are your testing efforts going? I’d love to hear about any challenges (and successes!) that you’re having.