Predictive Marketing

Always Be Testing

“It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.” – Mark Twain

In today’s rapidly evolving markets, you can never take anything for granted. The average lifetime of so-called “best practices” is shorter than ever.

I experienced this first hand recently on a couple of email tests designed to drive registrations for some upcoming conferences. Both conferences were targeted at highly technical IT audiences.

My past experience had always indicated that the best subject lines had offers and calls to action, especially when closing in on a limited time offer pricing deadline. So the following test results were as expected:


I had seen this result dozens of times before; invariably, the subject line that emphasized the dollar savings and created a sense of urgency had always emerged triumphant. Imagine my surprise, then, when I saw the results of the following test for a different conference directed at a similar audience:


Not only did the standard subject line offer no improvement on the alternative; it produced a result that was significantly worse. There is no doubt that, for this particular audience, the second subject line produced more conversions than the first.

The beauty of testing is that marketers don’t have to figuratively stumble around in the dark searching for the best way to communicate with customers. Test and learn strategies provide a way to find out directly, from prospects and customers, what they value and want most. There is no longer any excuse for a marketer to rely on hunches, anecdotes, and biased opinions in order to make marketing decisions. Even the seemingly most insignificant of decisions – the color of a registration button, for example – may have an effect on conversion rates which can be quantified.

However, when employing a test and learn approach to marketing, there is a trap to be avoided, which is illustrated by this example. Every business, and every customer set are different. There is no one set of best practices that apply in every situation. Think of the body of knowledge that you gain by testing to be a set of “best guidelines” rather than best practices. And know that no testing program ever arrives at a final best answer. Your customers are always changing. Always be testing.

Email Campaigns and Customer Engagement

Many marketers pay a lot of attention to email stats such as opens, click-throughs, unsubscribes, etc. When you have embarked on a campaign that involves multiple emails to the same list, one of the factors you should consider is how deeply you are engaging the universe of customers that you are trying to reach.

For example, it’s typical to see open rates that decline by a fairly constant percentage on each effort, such as 20% on the first email, 15% on the second, 11% on the third, and so forth. If you were to graph the percentage of opens as a function of the number of emails sent to this customer list, it would look something like this:

multiple-emails-to-same-list-graph-v2The question now is, how many unique names on this list have actually opened the email? Or, equivalently, what percentage of the list (i.e. market) has been engaged?

At one extreme, the minimum penetration would be 20%. In other words,after the first email, no one new is opening the email. The opens on the second through fifth efforts are coming from an ever decreasing proportion of the orignal 20% who opened the first email.

At the other extreme, the maximum penetration would be 61%. In this case, the prospects opening the email are different on every effort. No one opens any of the five emails twice.

As you might expect, the actual percentage of penetration is somewhere in between. Upon examination of the email addresses from the prospect list that opened up each email, the actual cumulative penetration is 33%. The penetration after each effort looks like this:

cumulative-market-penetrationSeveral facts are clear from this. First, the 33% penetration is a lot closer to the lower of the two extremes. Second, it is clearly approaching a limit. Finally, the limit in this case, judging from the trend of the graph, is surely less than 40%, even if additional efforts are made.

There are a number of ways to use this information:

  • To keep emailing this list with essentially the same subject lines, “from” field, etc. is like beating your head against a wall. Instead of continuing to beat your head against a wall, you can test new subject lines to try and motivate new prospects to open the emails.
  • If additional information about the prospects is available, by statistically analyzing the differences in the characteristics of those who opened the emails versus those who didn’t, a predictive model can be developed to devise a messaging strategy to motivate those who didn’t open the email to open subsequent emails with the new message.
  • This information can also be used in predicting which third party lists will be most responsive and in devising messaging strategies for other lists with similar characteristics.
  • Even if you find that you can’t penetrate this list (or market) any further, you can save time and money by not trying to do so.

These graphs may look very different for your lists. But the information you need to analyze a major list in this way is readily available, and you can use it to your advantage.

« Previous Page

Predictive Marketing