Today, I was speaking with a data-driven marketer who confessed that he could not quantify the value of his email campaigns last year, since they did not build in control groups. Often I find this situation repeated among our clients, usually due to one (or more) of three reasons:
1. "Senior management feels we need to mail everyone." When budgets get tight, executives who do not understand the need for learning will open the spigot and insist that every dollar counts.
2. "We have no time to set control groups." When timing is really tight, control groups may be foregone in order to just get the campaign out.
3. "Since we have no e-commerce, we cannot measure email anyway."
Each reason is valid. But the value of learning what works and what does not by using control groups consistently pays off, in the short term as well as the longer term.
Arthur Hughes, one of the gurus of database marketing, writes about the importance of control groups in his blog last month. The learnings are critical for any database marketer.
The Importance of Control Group
Success in the market is the only valid measure of our promotions.
What this means, in practice, is that your customer database must be riddled with control groups- people who are just like everyone else, except they don't get the same promotion that is being sent to thousands of others. Only in this way can we learn whether the promotion is worth while. Let me give you some great case studies.
Creating a control group is not easy. You probably have several marketers who are dreaming up promotions by both direct mail and email. When the promotion is ready and approved, they want to shoot it out to as many people as possible. Telling them to set aside a control group may just be annoying to them. "We are not doing market research, we are doing marketing!" they may exclaim. Wrong. We must all be in the research business if we want to keep our jobs. Someone in the organization must be in charge of coding some customers as being in control groups, and imposing the discipline on others to respect the control groups, and not to send their promotions to it.