'I Was Told There Would Be No Math.'

May 7, 2008

I am no longer satisfied looking at program dashboards that roll up multiple messages and present average/mean performance numbers without a measure of variance. This wasn’t always the case.

When I first cut my teeth as an email marketer in the late 1990s, it had been a while since I’d taken a math class. Having attended a small liberal arts school in the Midwest, I was never asked to crunch a single number, and happily concentrated on the subjects that most interested me at the time—Cultural Anthropology and Sociology. So when I went to report on those first campaigns, I didn’t have a whole lot of tools in my belt to analyze them. I calculated the basic performance metrics—open rate, click through, etc.—and when trends or anomalies presented themselves I took notice.

Since those good old days, I have brushed up considerably on my analytical skills. Now I still rely on many of the same KPIs, and a few more, to judge program performance, but I definitely see the data in different ways.

As mentioned above, now I can’t look at a set of data without calculating their standard deviation. It’s of course useful to know, for example, what the average open rate is for a group of messages, but without a measure of variance, the average doesn’t yield much useful information. For example, I could have two different campaigns of three messages each that both have an average open rate of 30%. If we stop there, we could deduce that both programs are performing similarly with regards to opens—when one program could be comprised of messages with open rates of {29%, 30%, 31%} and the other could be {3%, 19%, 68%}. The standard deviation of the open rates for the first campaign is 1—not much variance—but the standard deviation of the second is 33.87, indicating some substantial fluctuations. Again, both have mean open rates of 30%, but there are probably different things going on in data set #2 that warrant further exploration.

If you don’t currently calculate the standard deviation of your campaign metrics, I suggest you try it. Don’t concern yourself that the standard deviation is the “root mean square deviation of values from their arithmetic mean,” just type =STDEV into excel and select the data. It only takes a second, and I believe you may find the information useful.

—Nicholas Einstein of Datran Media

Comments (0) | Posted on May 7, 2008 10:28 AM

the voice of email

Welcome to the Email Experience Council's blog, a forum for the email marketing industry's leading voices. On these pages, you'll find the opinions and thought-leadership that's driving the next evolution of email.

feed sign-up

Subscribe via RSS or email.

newsletter sign-up

After subscribing to this blog feed, also sign up for the Email Experience Council's weekly newsletter, which contains information on the latest email marketing initiatives, research, news and events.

Become a Sponsor of the Email Experience Council

search this blog

May 2008

Sun	Mon	Tue	Wed	Thu	Fri	Sat
				1	2	3
4	5	6	7	8	9	10
11	12	13	14	15	16	17
18	19	20	21	22	23	24
25	26	27	28	29	30	31

the voices of email

The Email Experience Council's membership includes many of the brightest and most committed email marketing experts. We're pleased to have some of them share their insights here on these pages. Our blog contributors include:

Elie Ashery is the president and CEO of Gold Lasso, and is responsible for the company’s vision and strategy execution. Before joining Gold Lasso, he co-founded Newsletters.com in 1997, selling it to The Tribune Cos. in 2000. He then worked for IncenSoft, focusing on email marketing while there. Read more.

Amy Bills is the senior manager of field marketing at lead optimization company Bulldog Solutions. She is responsible for lead generation and the go-to-market execution of Bulldog's new products and initiatives. Amy was previously the editorial team leader of Freescale Semiconductor’s internal creative agency and a senior editor at Hoover’s Online. Read more.

Nicholas Einstein is director of strategic and analytic services at Datran Media. Specializing in email and CRM strategy, he helps some of America’s top brands leverage online channels to communicate more effectively with their customers and prospects.

Lisa Harmon is a principal at Smith-Harmon, a creative services consultancy dedicated to email marketing strategy and production. She works with marketers to increase clickthrough, maximize revenue, and infuse delight into their email creative. Lisa is also the blogger behind edm.smith-harmon.com, an ongoing commentary on the best (and worst!) in email marketing creative. Read more.

Chip House is ExactTarget's VP of marketing services, leading the teams responsible for client success. He was named to BtoB Magazine’s 2005 “Who’s Who in B-To-B,” for being a vocal proponent of legitimate commercial email and an active lobbyist regarding spam and privacy issues. Read more.

Stephanie Miller is VP of strategic services for Return Path, the leading email performance company. She works with marketers to earn a higher ROI and response from their acquisition and retention email programs—developing content, contact and segmentation strategies, along with testing, measurement and production programs. Read more.

Jeanniey Mullen is the eec’s founder and the global EVP and CMO of global online publishing company Zinio. She is a thought leader and visionary in the email and digital marketing field. A columnist for ClickZ, she has published numerous papers and is a frequent speaker. Read more.

Charles Stiles is the VP of worldwide business development at Goodmail Systems. In his role, Charles is focused on helping generate a better understanding of the email environment and potential solutions for a better consumer experience. He currently serves as the chairman for the Messaging Anti-Abuse Work Group. Read more.

DJ Waldow is an account manager at Bronto Software. He works with Bronto’s largest clients to help them achieve and surpass their marketing goals. An active member of the email marketing community, DJ posts regularly on the Email Marketer’s Club, publishes a bi-weekly email marketing best practices newsletter, and films BrontoFire. Read more.

Chad White is the EEC’s director of retail insights and editor-at-large. He founded and is the author of RetailEmail.Blogspot, a blog dedicated to tracking the email marketing practices of the largest online retailers. Chad regularly writes major research reports on email marketing and is an Email Insider columnist for MediaPost. Read more.