Reporting and Metrics – the open rate debate

Everybody has heard of open rates and click through rates and all ESP’s report them but what do these figures really mean in relation to the success of your campaign? As mentioned in the DMA whitepaper on Email metrics and measurement, Einstein once said “Not everything that can be counted counts and not everything that counts can be counted”. This is very true when talking about email campaign metrics. Can you judge the success of your campaign based on the open rate ?

Tim Watson recently wrote that “open rates are as useful as your appendix“. I agree with everything he says in his article and that open rates need to be understood and in context to have any useful meaning.  Increasing your open rate doesn’t necessarily mean a more successful campaign.

What does the open rate tell us and is it reliable? Let’s start by looking at things from a slightly different perspective. What metrics do we want to know about our email campaign? In general we want to know if the intended recipient received the email and was it of interest to them. Did they read any of it or did they delete it without looking at the content? Did they follow any calls to action in the message? Armed with this information we could make a better estimate about the success of our campaign.

These are difficult metrics for an ESP to collect. There are companies such as Litmus that can help with this using CSS techniques and streaming images but really, in the B2C world, it is only the ISP’s who run the webmail applications that can get the accurate stats. ISP’s gather lots of statistics about how mailboxes are used to try and measure user engagement and whether or not you want the email you are receiving. Included in these statistics must be whether you actively open an email or send it straight to the trash as well as many others.

The read rate of a campaign is really the holy grail of email marketing. Even ISP’s would find it very difficult to give you a truly accurate measure. Consider your own inbox in whatever webmail or email client you use.  If you select a message and the content loads in the preview window, does counting how long you are previewing this message accurately say how interested you are in the content? Possibly not. For me, the last email selected in my inbox is the last message I read before I became distracted doing something else. Everybody has different inbox triage but there is no way of knowing if the user is still reading the message unless there is some interaction. Do they scroll the window, do they click on a link, do they mouse over content? However many stats the ISP’s collect it is very unlikely they will make them available to ESP’s so we have to try and “best guess” the stats.

The only metric that ultimately matters in an email campaign is the goal that you set for success before the campaign is sent. Whether this be form registrations, product purchases, website traffic, telephone calls etc. Many people have differing views on whether open rates are a useful statistic as the great open rate debate shows. However there is one thing that everyone agrees on. A campaign should never be judged on its open rate alone.



This entry was posted in Metrics and tagged , on by .
Simon Hill

About Simon Hill

Simon is the co-founder of Extravision, a privately owned ESP based in Manchester and have been involved in email marketing for over 10 yrs. His role has been to develop the technology and product from the ground up to provide a stable and secure infrastructure. Today his current focus is moving more towards looking at how we deliver the emails and our “delivery reputation” as well as growing the business and investigating new technologies.

Before Extravision, Simon was development manager at Productivity through Software, a software house specialising in reselling and developing tools for software developers. They started using email as a marketing tool in the late 1990’s and Simon headed up the development of their first email marketing tools. Simon joined the DMA Email Marketing Council Best Practice and Legal Hub in May 2010.