You may recall last week we started looking at testing and my recommendation of ten mistakes to avoid. Well this week I am going to continue with this theme and look at the remaining 5 mistakes that we should all look to avoid with our testing.

However before we get started, lets take a second to remind ourselves of the five mistakes we discussed last week:-

• Mistake 1 – Not Testing At All
• Mistake 2 – Not focusing testing on the big things that are going to make a difference
• Mistake 3 – Not having a hypothesis to prove
• Mistake 4 – Not testing with a robust data sample
• Mistake 5 – Forgetting to keep a control group

Mistake 6:- Don’t draw your conclusion too quickly
Because we are often working against the clock, it is easy to fall into the mistake of drawing a conclusion from test results too quickly and deciding to use a particular subject line because it generated the best open rates within the first 6 hours of an email being sent.

This is a mistake because often the people who open the email quickly are the ones who will always open your emails.  If you were looking to increase open rates this is going to come by engaging those people who don’t typically open your email.  Allowing enough time between conducting the test and analysing the results is crucial to improving understanding.

Mistake 7:- Don’t forget to measure and use what you are testing
I feel almost stupid writing about this mistake as it does sound ridiculous that people would test but not measure success, but I have seen it happen.  Without having agreed metrics to measure your test against it is impossible to determine the results of the test and therefore draw any conclusions for the future.

The other common mistake along the same lines is to conduct a test and then forget to share or use the knowledge in the future.  Either not building what you have learnt into future campaigns or using the knowledge gained but failing to document why.  I have spoken to businesses that send their messages at 10:00 on a Friday morning because they tested it, but don’t now have access to the results to support that decision.

Mistake 8:- Testing too many elements at once
When you want to make big improvements to your email activity quickly it can be useful to test different elements at the same time.  This has the positive effect of dramatically changing the results of your campaigns.  One word of caution however is to not make the mistake of testing too many different elements in one go.  This is a mistake as it often incredibly time consuming or impossible to then work out what elements of the test were successful and what elements weren’t.  In essence although you have improved results, you don’t have the knowledge to help you understand your audience and inform future campaigns.

Mistake 9:- Allowing technology to determine success
I am aware that many technology systems will run a test and automatically send emails to the remaining recipients based on agreed success criteria.  This saves you the trouble of logging in to check the results, but it is a mistake.  Only you can really understand what has been learnt from the test and decide on the right approach for the future.  As an example, the subject line with the highest open rate might be determined to be the most successful, but this doesn’t necessarily take into account the fact that the other subject line generated greater click through rates and conversion levels or that the first subject line also created a lot more questions or complaints.

It can be a big mistake to not take the time to determine the results of the test yourself.

Mistake 10:- Don’t forget to transfer the insights from testing to other marketing channels
It is easy to think that the results of testing an email are specific to the email channel, however this is a mistake.  How a recipient interacts with your email will tell you a lot about what they are interested in and how they feel about your business and brand.

I would always urge marketers to share the insights that they learn from their email testing and apply it to the other marketing channels that they operate.  For instance, using subject line test results to inform pay per click bidding, or the layout of your email to inform the content of a landing page or printed catalogue.  It can be a mistake not to share your email insights across the other marketing channels.

Testing is an incredibly important part of any email program and should be scheduled as part of any marketers email activity.  Avoiding the mistakes made above will help to improve the value of your testing and therefore email activity even further.  It would be great to hear about other readers testing experiences (good and bad), so please let me know yours.

About the Author

James Bunting is Director of Strategy and Research at Communicator Corp. View the full bio here.

Related Posts

Event: Email customer lifecycle: List growth, May 22nd

Join us at the International Email Marketing Summit on May 16, 2012

Three stages to developing an email marketing strategy

5 New Year’s Resolutions for an Email Marketer

5 Ways Email Marketing Must Adapt to Remain Relevant