Welcome to lesson 3 of Email University. In case you missed it, here is lesson 1 and lesson 2. Also, you can join the private LinkedIn group here.
What to measure?
That’s the million-dollar question.
Every email campaign has a basic set of metrics — such as open-rate and click-through rate — along with a KPI, which varies based on the type of email you’re sending. The KPI for a survey might be completion rate, for a welcome email it’d be “confirmed email address” rate, and so on.
Notice that both examples use a rate — that’s extremely important. Using a conversion rate as opposed to absolute numbers helps you track performance regardless of email volume.
Let’s start by talking about the most commonly used email marketing metrics.
Basic email marketing metrics
In this bucket we’ll include:
Let’s explore these one by one.
List growth rate
If your list is not growing, it’s shrinking.
There’s a few reasons for this: readers will inevitably unsubscribe and a few might even mark you as SPAM. Also, older subscribers will become disengaged over time, leading to lower open-rate (vs. newer subscribers). If you need ideas on how to grow your list, check out lesson 1.
Unsubscribe rate
Most people cringe whenever someone unsubscribes — I rejoice. Think about it, somebody’s telling you they no longer want to receive your emails. They could very well stop opening them or mark you as SPAM but, instead, they took the time to say “hey, I don’t want this anymore”. There’s still engaged, and this presents an opportunity…
You should reach out to recent unsubscribes and ask why they did so. You might get feedback such as “you send too many emails” or “I just signed up for the ::enter free offer details here::”. This information is pure gold.
Here’s a sample email you could send:
Subject: Unsubscribe question
Hi first_name,
This is Carl from MailCharts. I noticed you recently unsubscribed from Email University and wanted to get your feedback. I’d love to understand why you joined and then unsubscribed. Did we sent too many emails? Were the lessons not valuable?
To thank you for your time and feedback, I’d be happy to send you a $5 Amazon gift card.
I hope to hear back from you soon — it’d mean the world to me,
Carl
P.S. This email was sent manually and it’s the last email you’ll receive from me.
Feel free to remove the gift card option or bump it to $XX depending on what makes sense for you. You should also adapt the tone of the email to match your company’s — notice how my writing tends to be conversational.
Open-rate
Everyone is obsessed with open-rates and rightfully so: if subscribers don’t open your emails, there’s no way they’ll get value out of them. A few things to note about open-rates: While most opens happen within a few hours of sending your email, some subscribers will open (or re-open) the email months later.
As mentioned earlier, a list’s open-rate (usually) decreases as the list’s subscribers age. It’s important to continually add new subscribers and to scrub your list by segmenting or removing inactive subscribers.
Below you’ll find the open-rate data for Email University lesson 1. Notice the spike right after the email was sent:
And notice how we’re still seeing opens days later:
Here’s how our first two emails performed for lesson 1 and lesson 2:
Noticed the open-rate went down from 51% to 41%? While the drop is drastic (-20%), this is expected. Our next email might have an open-rate of ~36%, the next one ~33% and so on. (Note: the last 2 numbers are complete guestimates.) Decreasing open-rates are painful, but normal.
Pro-tip: A/B test your email subject line. You’ve heard this before, but here’s why: you can double your open-rate by testing subject lines.
The hypothesis:
Our subscribers are more likely to open an email with a descriptive subject line vs. an academic one (like we had in the announcement for lesson 1).
The two subject lines:
- “Email University: Lesson 2”
- “45+ email examples you can’t live without”
Note: The final open-rate ended up being 41.89%. The test was only live for 1hr before a winner was picked. We set the test this way because we wanted to hit the entire list before 9:00 – 9:30am EST (when most of us get into the office). Depending on your business, you may want to let the test run for 2-3 hours to gather more data.
Click-through rate (CTR)
This one’s tricky. Most marketers say that a higher CTR is better. I say it depends.
Example: Imagine you send a daily weather report. You could include all of the information in your email, resulting a ~0% CTR. Alternatively, you could add a big “click here to view the weather report” button, resulting in a ~XX% CTR. While the later yields a higher CTR, it also translates into a worse user experience.
Don’t obsess on your CTR unless it makes sense to do so.
Another note on CTRs: look where people are clicking and remove unused links. Here’s the click report from our first lesson email:
A few things to question when you go through your click data: Should you keep your social profile links in your emails? Is your main CTA high enough? What about your email length — are they too long? Are people clicking on links towards the bottom? If not, test sending shorter emails — these tend to have a higher CTR.
When it comes to email metrics, it’s important to keep in mind that there’s so much you won’t know about. As an example, in lesson 1 we offered a free Udemy coupon. Here’s the click activity for it:
By the looks of it, you’d think that 69 people clicked on the link. Let’s image that the 83 readers that viewed lesson 1 in their browser also clicked on the Udemy coupon link. This gives us a total 152 clicks to the Udemy coupon.
Based on this data, we’d expect a maximum coupon redemption of 152 (assuming 100% of readers claimed the coupon and signed up for the course, which is unlikely). To simplify the math, let’s assume we haven’t done any public distribution for this lesson (which we haven’t done yet) and that the coupon link was not publicly shared (we couldn’t find it in the open, let us know if you do).
We issued 10,000 Udemy coupons and expected a max redemption of 152… Let’s now look at the Udemy data:
Whoa, there’s only 8,860 spots left! That’s right, we saw over 1,100 coupon redemptions. (10,000 – 8,860 = 1,140) That’s 700% higher than we expected.
While this example is extreme (it’s not every day that you can offer a $297 freebie), this should make your spidey sense tingle. Measure what you need but remember that when it comes to email marketing, there’s a ton that will happen that you won’t know about (e.g. I forward your email to my team of 20, or 200, or 2,000.)
Pro-tip: Re-send your email campaign to everyone that did not open it the first time, with a different subject line, a week later. Shoutout to my buddy Noah for this one.
SPAM complaints
There’s nothing much to say here other than don’t SPAM people and make it easy to unsubscribe. If you use overly promotional material, or spammy looking subjects lines such as “FREE XYZ JUST FOR YOU!!!!!” you’re likely to hit SPAM filters.
If you need help with SPAM testing, check out Litmus.
My favorite metric
Now that we’ve covered the basics, let’s talk about the metric I care about the most: response rate. Just about every email marketing strategy I create includes at least one email where I ask for a “reply”. Allow me to explain.
About me: I work full-time at Thinkful, where I lead Inbound Marketing. On nights and weekends, I’m building MailCharts with my buddy Tom. At Thinkful, my goal is to build an audience and, in order to do so, I need to understand what topics prospective students care about. For MailCharts, we’re still looking for product-market fit.
In both cases, we need to talk to prospective customers to understand their needs and what they care about. In order to do this, as soon as you join the email list you get an email from me asking something about you. What I ask is different for each company or list. As an example, here’s the one for Email University:
I wanted to know more about you and your email marketing challenges.
How do I measure the success of these emails? Response rate. Get this number by counting how many unique email responses you’ve received (straight from Gmail) divided this the number of emails you sent (get this from your ESP).
This particular email received a 2.3% response rate. My max response rate has been around 10%. Note: This varies drastically based on the target demographic, why someone joined the list and the traffic source (how someone found out about the landing page where they joined the list).
Pro-tip: Subscribers are never as likely to open your email, and engage, as they are right after they sign up. Send this type of email early on (as in, minutes after they join).
If you need some inspiration on which questions to ask, read the article below:
What’s the biggest pain in how you work? If you could wave a magic wand and change anything about what you do, what would it be? source
Bonus: Metrics-driven optimization for email drips
Our friends over at Vero blogged about this and it’s genius. Here’s what you do: Take all of the emails in your drip sequence and plot them on a chart to see which email resonates the most — and least — with your subscribers.
Here’s the result (interactive graph here):
Any email above the trendline is doing great. Any email below the trendline could use some love.
Let’s walk step-by-step on how to do this.
Step one
Enter your data on a Google Doc and select it.
Step two
Insert a Scatter Chart.
Step three
Add a trendline.
Trendline note
In our example, we selected a Polynomial trendline to smooth out the curve.
Here’s a bit more about polynomial trends (disclosure: I don’t fully grasp every detail related to polynomial trends. If you do, please tweet me @carlsednaoui).
Here’s what our graph would look like if we used a linear trendline — the spike in click-rate at the end is not fully reflected.
Compare this to the polynomial trendline:
It’s your turn: Action item summary
- How fast is your list growing? What else could you do to make it grow faster? Check out lesson 1 for inspiration.
- Reach out to everyone that unsubscribes from your list and ask them why.
- A/B testing subject lines can drastically improve your open-rate. Run A/B tests to understand what resonates with your subscribers.
- Don’t obsess on your CTR unless it makes sense to do so. Oftentimes, a lower CTR is ok if that translates in a better user experience.
- Test sending shorter emails and measure the impact on your CTR.
- When it comes to email marketing, there’s a ton that will happen that you won’t know about.
- Re-send your email to subscribers that did not open the first time, with a different subject line, a week later.
- Engage with new subscribers and get to know them and their needs. Keep track of your response rate.
- Analyze your email drip performance and see which emails could use some love.