Chapter 10 – Measurement

Back in the second chapter of this book, I used the example of a fictitious retailer known as Duffy’s Depot to illustrate lifetime value.  Now we’re going to get into some of the nuts and bolts of measurement, continuing to use the Duffy’s Depot example.

This is not a book about measurement.  I’m going to review enough about measurement to get you started.  If you have some analytical training or experience you may not need much more than what I’m going to share with you.  If you’re an experienced analyst in the world of direct marketing you may find my examples to be fundamental.

Lifetime Value as Measurement

Lifetime value itself is not a measure of success.  But changes in lifetime value can be a measure of success.  The dilemma is this.  Lifetime value is a long-term concept.  Remember the example we used before?  We were looking five years into the future and discounting expected cash flows to present value.  It’s an important concept to review with others in your company, but the fundamental reality of today’s business climate is that often programs must pay for themselves rather quickly.  We need to look at ROI.

Control Groups

Before getting into ROI, we must have a quick review of control groups.  If you know what a control group is, skip this.  Otherwise, read on.

A control group is used to isolate the impact of a marketing program or group of marketing programs.  A control group is a random group of like customers held out of the marketing program.  Behavior of customers in the control group is compared to the behavior of the customers in the marketing program.  Generally all other things are considered to be equal, so the differences in behavior between the two groups are considered to be attributable to the program.

Here’s a simple example.  Let’s say we’re executing a direct mail program within our contact strategy.  We select customers in the highest spending segment because we’re going to send a special recognition mailing.  There are 100,000 customers in the segment.  Before executing the mailing, we randomly extract 10,000 of those customers to be our control group (groups much smaller than 10,000 can raise issues about statistical validity).  The control group customers are similar in make up to those who will receive the mailing.

Let’s say we analyze spending behavior during a period of time following the mailing.  I can’t tell you absolutely how long a period of time you should analyze.  You should look at response levels each week after the mailing occurs and determine the point at which there no longer appears to be a difference between customers in the mail group and those in the control group (there are more rigorous statistical techniques for getting precise about this).  Suppose we conclude that the effectiveness of the mailing ended after four weeks, and the total spending in the two groups is as follows:

We want to determine how much incremental spending occurred.  To do so, we first must adjust these numbers.  We must project what the control group spending would have been if it were the same size as the mail group.  Remember that we had 100,000 customers to start with.  We held 10,000 out as our control group so 90,000 received the mailing.  In this case I simply multiply the control group sales by nine to make it an apples to apples comparison with the mail group.

Here are the adjusted numbers:

That’s the concept of a control group and how it’s used.

Return on Investment (ROI)

Let’s first take a look at calculating ROI on a contact strategy.  We’ve been through the control group explanation so that gives us a nice head start.  With a contact strategy it’s best to hold out a control group for the entire period you’re testing the contact strategy (six months to a year).  It’s best to have a control group from each segment or, if you select one control group, at least examine it to ensure that it represents each of your segments proportionately.  For instance, if a particular segment represents about seven percent of your customer base, it should represent about seven percent of the control group.

Once you’ve completed your test period, calculate the incremental spending as we did in our control group explanation.  Convert incremental spending into incremental gross profit (or incremental contribution margin, if that’s an approach more commonly used in your company).  We’re trying to determine the portion of incremental spending that goes to cover fixed costs and profit.  Let’s assume we’re going to use gross profit and it’s at a rate of 35%.

The next component we’ll need to calculate ROI is the amount we invested to make the results happen.  It’s the budget we applied toward our segmentation and contact strategy to generate the incremental gross profit.

Let’s assume the following:

The ROI is calculated as (Incremental gross profit – Budget spent)/Budget spent.  This tells us that we put $250,000 into the segmentation and contact strategy.  That’s the budget we spent.  It’s the investment component of return on investment.  We generated $350,000 in incremental gross profit.  That paid for our $250,000 investment with $100,000 to spare.  The $100,000 is 40% of the $250,000.  That’s ROI.

Promotional Currency ROI

Before jumping into an ROI calculation, we must consider some of the promotional currency idiosyncrasies.  Let’s assume we have a program at Duffy’s Depot in which members earn five points per dollar spent.  Once a member earns 500 points, a five dollar reward is issued.  This looks like a program funded at a rate of five percent.

This apparent five percent funding rate is what I call a gross funding rate.  To calculate our real cost, we must convert the gross funding rate (GFR) to a net funding rate (NFR).

The first thing we must adjust for is breakage.  Breakage is the currency never redeemed.  Over time you’ll get a sense of what the breakage will be in your program.  Breakage typically ranges from 20% to 60%.  For our calculations, let’s settle on 40% breakage.  If 40% of our currency is not redeemed, 60% is redeemed.

The second thing we must adjust for is cost of goods sold.  This assumes we’re giving away our own product as a reward.  (If that’s not the case, you’ll need to adjust based on the actual cost of the rewards you’re giving away.)  Cost of goods sold is (1 – gross profit).  We’ve been using a gross profit of 35%.  So our cost of goods sold is 65%.  When we sell something for one dollar, our cost is $.65.

The NFR = GFR * percent of currency redeemed * cost of goods sold.

Using this example, NFR = 5% * 60% * 65%.

NFR = 1.95%.  That’s the funding rate after adjusting for actual redemption and cost of goods sold.

Let’s walk through an ROI scenario with our NFR.  We’re still using the same program scenario for Duffy’s Depot.  Here are some additional assumptions and a calculation of ROI.

·        NFR of 1.95%

·        Communication cost of $1.50 per member, per year (assumes we’re using e-mail effectively)

·        Infrastructure cost of $500,000

·        500,000 members

·        Sales lift of 15%

If you’re interested in calculating the breakeven point, simply adjust the incremental sales number until the ROI becomes 0.  That looks like this:

In this scenario it requires a 6.66% lift in sales to breakeven.

Research for Measurement and Insight

Everything we’ve looked at so far is linked to changes in behavior.  We should not overlook changes in customer attitude.  Behavior is a clear indication of what’s happening now.  Attitude is an indication of what’s likely to happen in the future.

If you have been able to maintain a control group, you can conduct sample research on customers in your program versus those in your control group.  If you have not been able to maintain a control group, you can conduct sample research on those customers in your program versus those who are not.  This second method has some obvious flaws in terms of bias.  But it’s better than nothing.

Qualitative research is good for getting directional insight into your program and feedback from customers.  Quantitative research can uncover signs that your program has impact beyond just what you can ascertain from behavior measurement.

The following are some things to look for in your research.  In each point I’m referring to differences between customers in the program versus the control group, or between customers in the program versus not in the program.

·        What brand do members mention first in your category?

·        When prompted with a list of choices in your category and asked to rank order the choices, where do members place your brand?

·        How do members rate your quality, value, price, service, selection and convenience versus your competition?

·        How likely are members to complain rather than quietly defect?

·        How likely are members to refer friends and relatives to your brand?

This is a start.  If you have a regular tracking study you do with customers I suggest segmenting it based on membership (members versus control group or members versus non-members).  Sample research is an important way to augment your behavior measurement.  It provides additional texture about what your loyalty efforts do for your brand.