When using PowerMail's built-in A/B testing tool, results now include whether or not the results are statistically significant.
What does this mean?
Statistical significance is a way of saying that the results of a test are important, that they have meaning which is a reflection of real world results and are not just the result of randomness. Having statistically significant results to a test is great! It means that your test, be it subject line, time of sending, etc. has a result which shows real, verifiable meaning.
Why are my results not statistically significant?
The largest factor which influences statistical significance is the size of the sample group.
Say you sent out an A/B test on subject line to two groups of 250 each. Group A had 75 opens, an open rate of 30%. Group B had 87 opens, an open rate of 35%. That’s a 5% difference. Looking at the numbers alone, that seems like a pretty significant difference, right? Unfortunately, not really. Because that 5% only represents 12 people, it’s not a large enough number to have any real significance and could be completely random.
Now, if 100 people from Group A had opened the email, it suddenly becomes very significant and there is less than a 1% chance that the results are random and slightly more than a 99% chance that the results are accurate. The difference in these open rates is large enough to have a statistical significance.
| Want to try running these numbers yourself? This A/B significance calculator is a pretty good place to start. |
But what if my email list is too small to generate statistically significant results?
One option is to focus less on A/B testing of your entire email list, and instead focus on list segmentation. When sending one message to your entire list, you risk sending people irrelevant content. Recently, we sent out an email about a product upgrade that only affected a small number of our clients. That email, with an open rate of 60%, exceeded a typical email blast. If we had sent that email out to our entire list, not only would few people open the email, but we probably would have lost a lot of subscribers because we were not delivering relevance.
| Sending people content they want - and only the content they want - can increase open rates and reduce unsubscribes. |
An effective technique to reach only the people who will find your email relevant is to use segmentation. List segmentation means you are creating specific, targeted email lists based on certain criteria. The criteria you use will depend upon your organizational goals and needs. Using multi-step search queries, you can slice and dice your email list into smaller, targeted lists of individuals with similar interests and behaviors.
Organizing your email segments
Once your Databank has been segmented into smaller groups, it’s time to use Subscription Management to create small, meaningful email lists. By using Subscription Management, you give your members the ability to determine the content they find relevant.
Comments
0 comments
Please sign in to leave a comment.