Category Archives: Metrics

 

Are high open rates holding you back?

Our findings last month on the Obama campaign caused a lot of debate but the bare facts of our analysis still stand – had Obama’s team optimized for improved open rates, their send volumes would have dropped and their all-important donations would have followed.

Open rates remain a widely used and hugely misleading measure of performance and engagement in the email industry. At best they give you an idea of a campaign’s performance in isolation but at worst they lead email marketers to focus on optimizing the wrong strategies for their email program.

Here we discuss how to identify if maximizing open rates is holding you back and how to go about identifying the strategies that will have the biggest impact on your results.

The open rate paradox

Using EDS Analyst we examined the relationship between open rates and total unique opens for the top 200 email senders by list size in the US for 2012.

We were confident that, like the Obama campaign, there would be an inverse relationship between rates and totals – so as rates increase, totals decrease and vice versa. We call this the open rate paradox or to paraphrase a popular sports trusim: rates are for show, totals are for dough.

Each dot on the graph below represents a single sender and we picked out some well-known brands as reference points.

Sure enough, the graph shows that for most large senders, there is an inverse relationship between open rates and the total number of opens – the higher the open rate, the lower the number of total opens. Rates are for show.

It’s also no coincidence that nearly all of the brands with the biggest lists (orange dots) also have highest number of total opens because they are sending more opportunities to open.

Although opens don’t directly correlate to revenue, even the most avid fans of open rate maximization would agree that the more people that actually open your emails, the more engaged your database and the more revenue or conversions you are likely to generate. Totals are for dough.

Keep it simple – focus on just three strategies

If your goal is only to improve open rates, then your strategy is simple: halve your list by suppressing your less active subscribers and watch those rates soar… and those total opens plummet! But if your goal is to increase total opens, then the bell curve in the graph above helps define three clear strategies:

  1. List size:
    Has the biggest impact on totals and can be improved independently of the other two.
  2. Increase send volume:
    Significantly increases total opens for relatively little effort (low effort to gain ratio).
  3. Optimize for rates
    Increases total opens but requires the biggest effort (high effort to gain ratio).

Most brands are clustered towards the lower middle of the curve because it’s the easy place to be. By and large, they all put a similar amount of effort into their program and use the same undefined strategies.

The outliers, however, go above and beyond in one of three ways – those to the right have very high open rates, those to the left have high send volumes and those at the top are combining high send frequency with very big lists to produce massive send volumes.

In effect, this is the three different strategies implemented to their extremes.

Of course, there are limits to the effectiveness of each strategy and these are defined in the graph above by the orange line to the left (frequency cap) and green line to the right (optimization cap).

These boundaries exist because for any given list size there is point at which diminishing returns kick in for both frequency and open rate. And, as the big empty space to the right of the green optimization cap shows, it’s very hard to send a large volume of email while still achieving a high open rate.

So the basis of a successful email program is to continually grow your list while finding a balance between increasing send volume and maximizing open rates with better offers, targeting, subject lines, etc.

And you find that balance by ignoring your open rates…

Define your strategy by ignoring open rates

To illustrate the effect these strategies have on an email program, we have created a simple optimization chart, below. The green curves represent the impact of send volume on total opens and the brown lines represent the impact of open rate on total opens.

Each intersection represents a hypothetical 10-hour unit of resource, as a means of comparing the effort required to implement each strategy. As you get closer to each cap, the effort required to improve your totals with your chosen strategy increases exponentially.

Imagine your brand is the star in the middle of the curve and you want to take on your leading competitor, the lightning bolt.

If you use open rates to define your strategy, then you focus your resource on maximizing those, route A. Your open rate may now be much better than your competitor’s but they are out-mailing you, so they are still creating twice as many opportunities to buy or convert.

If you choose to increase your send volume, ‘route B’, then your open rate drops but your total opens more than double. However, as you approach the frequency cap, the impact of your strategy diminishes and you still trail your competitor.

If you use totals to define your strategy, then you take ‘route C’, which balances resource between increasing send volume and maximizing open rates. Your open rate drops but you are finally creating more opportunities to buy than your competitor.

Smart email marketing is not just a case of increasing send volume indiscriminately or of only focusing on ever tighter targeting. There is a balance that exists for each brand, you just have to find your own sweet spot.

Total opens the key to optimizing your program?

In this instance, we have highlighted the open rate paradox using total opens because that was the data available. However, we’re confident you will find the same inverse relationship in your own campaigns with total clicks and, more importantly, revenue. And in the end that’s the only metric that matters!

 

Email engagement, often discussed now defined

Everyone wants it, but there is no industry consensus on the best way to measure it. I’m talking about engagement in the email channel.

Take an example, a fashion brand might send two or three emails per week. It’s not realistic to expect that most people are going to be interested in buying a new fashion item every week or even to review offers each week.

Just because someone is not in the mood to buy or look at current offers does them make them no longer engaged with a brand? Of course not, they gave permission to receive the emails, they showed engagement, ignoring a few emails does not mean a lack of engagement.

Classically campaign open and click rates are used to judge engagement. This was fine when brands sent one campaign per month. Email volumes have increased considerably in the last five years but metrics have not moved on.

A re-think is needed as the classic metrics measure campaigns not customers and as a result promote the wrong behaviour in email marketing.

Its customers that need to be engaged so measuring campaigns makes no sense, its customers that should be measured.

I’ve been working on a paper, along with my fellow DMA Email Council hub members, Dela Quist, Skip Fidura and Kath Pay. The paper goes to the core of how to measure customer engagement in the email channel and delivers a verdict, based on analysis of brand data.

The paper has been put together to kick-start the discussion in the email industry about just what should be measured and the debate is starting at Email Evolution 2013 conference in Miami this week. For DMA blog readers we’re releasing the paper to you ahead of the event.

I hope you download the paper and find it thought provoking. If you leave a comment one of the paper authors will reply.

The truth behind the buzz: what really made the difference in Obama’s re-election

One of the big stories in digital marketing in recent months has been about a campaign whose results have a major impact on people around the world – the US Presidential Election. Marketing pundits representing all channels have had an opinion on how Obama’s campaign led to his re-election but I would argue none was more significant than the use of email.

Let me start by saying there is no doubt this election was won by email. Here is a direct quote from an article published by Business Week; “Most of the $690 million Obama raised online came from fundraising e-mails”. And by the way,  $690 million represents nearly 75% of the $934 million raised in what ended up being the most expensive Presidential election in US history. This makes email, by far and away the No1 non-political contributor to the drubbing of Mitt Romney.

So why has there been so little said about this incredible achievement by email marketing pundits, ESP’s or their PR machines? When you look at the fuss made over the 2008 campaign – allegedly won by social media – the silence from the email industry has been deafening. When anyone does mention it, there has been a tendency to attribute the success of Obama’s email fundraising activities to anything other than email.

Some may suggest that without the “age old lessons [and presumably wisdom]” passed down from wise-old DM to rather awkward, gauche and somewhat unattractive email marketing the story would have been very different.

Nothing could be further from the truth. Here is what really happened. Obama won because he sent more email to more people more often than Romney period!

According to numbers put out by eDataSource and Return Path, Obama mailed a staggering 40+M subscribers compared with Romney’s 4M, on some days they sent 350M compared with 26M from Romney. So while relevance, engagement, creative – ugly or otherwise, Subject Line testing etc. did play some part in his success, they pale into insignificance compared to the impact reach and frequency had in his success.

What seasoned email marketers might find surprising (I see this as further proof of the fact that frequency drives engagement) is that the Obama database was more engaged and less likely to view the emails they received as spam. The figures below which I extrapolated from numbers published by eDataSource and Return Path illustrate this clearly:

Obama Read 15.85% Total 6,340,000
Romney Read 7.94% Total 317,600
Obama Delete Unread 9.01% Total 3,604,000
Romney Delete Unread 5.11% Total 204,400
Obama ISP Spam 17.95% Total 7,180,000
Romney ISP Spam 52.51% Total 2,100,400
Obama User-Marked Spam 0.02% Total 800,000
Romney User-Marked Spam 0.03% Total 120,000

 

The Obama campaign raised an average of $17.25 per subscriber, if you assume Romney was able to do the same, he would have generated $69M from email compared to Obama’s $690M. So if you were Romney what would you have learned from this, A) Segment and test your way to $172.50 per subscriber or B) Send email to a lot more people more often?

Email delivers something DM cannot. Broadcast reach at near zero marginal cost.

If you don’t want to leverage that, stop sending email!

I have read lots of peoples’ take on the article and what they found most interesting and would like to share mine – something Bloomberg Business Week chose to call a counterintuitive. I don’t and I think it is an awesome admission: “Most people have a nearly limitless capacity for e-mail and won’t unsubscribe no matter how many they’re sent”. Now read the comments made by the very people who made the campaign successful. Note how few said they enjoyed the blitz yet on average they donated $17.25 each.

Now that’s an insight!

I know for certain that they are not the first people to have worked this out, but they are the first significant entity to come out and say it.

Let’s give credit where credit is due in the 2012 presidential election; segmentation, targeting and testing techniques were the tail, reach and frequency were the DOG!

Reporting and Metrics – the open rate debate

Everybody has heard of open rates and click through rates and all ESP’s report them but what do these figures really mean in relation to the success of your campaign? As mentioned in the DMA whitepaper on Email metrics and measurement, Einstein once said “Not everything that can be counted counts and not everything that counts can be counted”. This is very true when talking about email campaign metrics. Can you judge the success of your campaign based on the open rate ?

Tim Watson recently wrote that “open rates are as useful as your appendix“. I agree with everything he says in his article and that open rates need to be understood and in context to have any useful meaning.  Increasing your open rate doesn’t necessarily mean a more successful campaign.

What does the open rate tell us and is it reliable? Let’s start by looking at things from a slightly different perspective. What metrics do we want to know about our email campaign? In general we want to know if the intended recipient received the email and was it of interest to them. Did they read any of it or did they delete it without looking at the content? Did they follow any calls to action in the message? Armed with this information we could make a better estimate about the success of our campaign.

These are difficult metrics for an ESP to collect. There are companies such as Litmus that can help with this using CSS techniques and streaming images but really, in the B2C world, it is only the ISP’s who run the webmail applications that can get the accurate stats. ISP’s gather lots of statistics about how mailboxes are used to try and measure user engagement and whether or not you want the email you are receiving. Included in these statistics must be whether you actively open an email or send it straight to the trash as well as many others.

The read rate of a campaign is really the holy grail of email marketing. Even ISP’s would find it very difficult to give you a truly accurate measure. Consider your own inbox in whatever webmail or email client you use.  If you select a message and the content loads in the preview window, does counting how long you are previewing this message accurately say how interested you are in the content? Possibly not. For me, the last email selected in my inbox is the last message I read before I became distracted doing something else. Everybody has different inbox triage but there is no way of knowing if the user is still reading the message unless there is some interaction. Do they scroll the window, do they click on a link, do they mouse over content? However many stats the ISP’s collect it is very unlikely they will make them available to ESP’s so we have to try and “best guess” the stats.

The only metric that ultimately matters in an email campaign is the goal that you set for success before the campaign is sent. Whether this be form registrations, product purchases, website traffic, telephone calls etc. Many people have differing views on whether open rates are a useful statistic as the great open rate debate shows. However there is one thing that everyone agrees on. A campaign should never be judged on its open rate alone.

 

 

Valuable Insights in the latest National Email Benchmarking Report

The National Email Benchmarking Report for the second half of 2011 has recently been released by the DMA and it once again provides marketers with valuable insights into trends and challenges within the email industry.

Join our good friend Mark Brownlow of email-marketing-reports.com as he delves into the issues facing email marketers and how they have changed over the years that this report has been produced.

The data for this report is sourced from a host of Email Service Providers representing the majority of ESP-sent volume within the UK, and features both quantitative and qualititative responses. Also, for the first time, we are able to present sector-level data for the Retail, Finance, Travel, B2B and Publishing industries!

As I’ve previously written about, these statistics are good as a starting point or yardstick for your programme, but ultimately marketers need to ensure they are tracking interactions as far as they can, and also fully understanding the metrics they are seeing. A great resource to use in combination with this year’s report is the whitepaper on Email Metrics & Measurement, authored by the experts which make up the DMA Email Marketing Council’s Legal, Data and Best Practice Hub – that way you can see what the trends are in the industry, identify how your own challenges compare, and begin to benchmark & prioritise your own programmes to maximum effect.

Perhaps you’ve laid awake at night thinking…

  • Are people sending more or less email?
  • Is there greater focus on segmentation, or has batch-and-blast made a comeback?
  • Is Deliverability still an issue?
  • How do response rates compare in different industry verticals?

Find out the answers to these questions and more in the National Email Benchmarking Report 2011 (2012 Edition)!!

Join us at the International Email Marketing Summit on May 16, 2012

Register now for this virtual summit and learn all about the latest trends and best practices in email marketing without leaving your desk!

And it won’t cost you a penny/eurocent/dollarcent/… 

The DMA is proud to be a sponsor of this, the very first edition of the International Email Marketing Summit.

Not only will you be inspired by the latest tactics that work but you’ll also take away a list of action items you can implement immediately.

Featured speakers

  • Dela Quist, Alchemy Worx
  • Dave Chaffey, Smart Insights
  • Tamara Gielen, Plan to Engage
  • Denise Cox, Newsweaver
  • Riaz Kanani, Alchemy Worx
  • Kath Pay, Plan to Engage
  • Arianna Galante, ContactLab
  • Tom Bailey, eCircle
  • James Bunting, Communicator
#IEMS speakers

What’s on the agenda?

  • Beyond just selling: engaging with your subscribers
  • 7 reasons why your subscribers don’t respond
  • Tips & tricks for designing emails for a mobile audience
  • Inactive Subscribers: Prospects or Problem?
  • Creating a successful content strategy for email marketing: 8 Easy Steps
  • and lots more…

2011 roundup of best practice white papers – Chairman’s summary

The Email Marketing Council’s Legal Data & Best Practice (LD&BP) hub has been reviewing the current email marketing best practices document over the past few months, and the publication of a revised version is imminent.

One of the things that the review process has identified is a need for more detailed guidance in certain key areas of the email marketing customer life cycle. For this reason, a number of supporting white papers have been produced, which can be found in the “Toolkit” section of the DMA’s website (www.dma.org.uk/toolkit), where they are available for download free to Members.

Here’s a quick summary of what has been produced to date:

Deliverability: Aimed at email program owners who have realised that their broadcasts are experiencing delivery problems, and are trying to identify why this may be the case. Looking at key factors such as sender reputation, spam filtering, blacklist operators, the document provides common-sense guidance on how to deal with them, including 10 easy-to-follow steps to improve your email deliverability.

Creative: Good creative is still an important determinant of a successful email campaign, and is sometimes the only connection a subscriber has with your brand. This document demonstrates that email creative is not a dark art requiring witchcraft and technical know-how! Rather, in non-technical language, it provides some easy-to-implement recommendations that will quickly optimise the performance of your email campaigns.

Data Analysis & Segmentation: Sets out a simple process to help email marketers start segmenting their data, and analysing their results. It defines five key areas to focus on, including: setting objectives; finding the right data; choosing the right segments; different segmentation models, and; effective use of segmentation. It also examines the best methods and approaches to implementing segmentation, as well as how best to interpret the results.

Split Testing: Provides email marketers with the basic capabilities that they will need to run split-testing activity. It looks firstly at the fundamentals that need to be in place to run a split testing program, and then examines ten prime opportunities where split testing can be introduced into any email marketing program to identify the optimal approach to maximise campaign response rates.

Triggered Campaigns: Delivering timely and relevant email messages, using trigger-based email marketing, plays an important part of email best practice. By analysing subscriber behaviour and identifying meaningful changes and/or events, organisations can communicate with their customers at a point when they are most likely to be receptive. This strengthens customer relationships by making them feel valued, and it is not unusual for trigger-based emails to attract high open rates as a result.

In addition to the documents that have been described above, there are also three new white papers whose publication is imminent:

  • Using 3rd Party Data For List Rental & Lead Generation
  • A Layman’s Guide to Email Marketing Law
  •  Email Lifecycle Marketing

And there are a further two which are scheduled for arrival during Q1 of the New Year:

  • Organic List Growth
  • Measurement & Reporting

The production of these documents is a collaborative process and the Email Marketing Council, as the representative body of the much larger interest group, is constantly feeding in new ideas about key issues which email marketers would like to have expert guidelines for. Hopefully, the documents described in this article are servicing this need, but it would be great to have direct feedback on whether they are useful, and what the email marketing community would like to see produced next. If you have any feedback for us, then drop a line to email@dma.org.uk , or online via LinkedIn, Facebook or Twitter.

 

Guy Hanson Chairs the The Email Marketing Council’s Legal Data & Best Practice (LD&BP) hub. He is Director, Response Consulting for Return Path.