The Problem With Social Media Case Studies

AugieRay1
Augie Ray Director - Global Voice of Customer Strategy, American Express

Posted on April 4th 2014

The Problem With Social Media Case Studies

We all love a good case study, don't we? Especially in the social media business, where results can be difficult to measure and prove, there is nothing more validating than knowing someone else has succeeded and can show us the way.

Problem is, despite years of experience and thousands of case studies promoted by agencies, consultants and speakers, most companies are still struggling with their social media strategies. How is this possible when there are so many good case studies to be found?

One problem with case studies is the way marketers use them. There has been a tendency to latch onto any successful case study as an archetype for success, but case studies are not best practices. For example, I can present a case study on how dropping out of high school led to success for people like Walt Disney, Sir Richard Branson and David Karp, but that does not make skipping school a good idea. The same is true for many social case studies; social media professionals may love to learn how Coke or Starbucks succeed in social media, but how many brands can leverage the existing marketing spend, reputation and consumer affinity of brands such as those?

Several years ago, a consultant was pitching my team at a financial services company and offered Dell Outlet's Twitter strategy as a relevant case study. I stopped her and asked how Dell's use of Twitter to sell refurbished hardware was pertinent to us in the banking and insurance business. The consultant was unable to answer that, and it made me wonder how many times that case study had caught the attention of marketing professionals who had absolutely no opportunity to use that knowledge within their own business or vertical. Who cares how much money Dell Outlet made on Twitter if your company cannot promote reconditioned product via a Twitter account?

Aside from how they are used, many case studies suffer from one or more problems that limit their value. First, most are produced to promote an organization's or person's products or services. This is not necessarily a bad thing, but it should cause us to approach the information with a healthy dose of caution and to question if we are getting the full and complete story.

Second, most case studies, particularly in social media, do not close the loop in terms of both costs and benefits. It is rare to see case studies with dollar signs--either for investments or returns--and this means most case studies are less business examples than they are entertaining stories. After six years dedicated to social business, I cannot stand to see one more case study measured in new fans or retweets. (And neither should you!)

My greatest concern with case studies, however, is that most simply do not hold up to any sort of scrutiny. Hungry for evidence of social media success, marketers have been far too eager to drink the social media Kool-Aid.

Take, for example, Dell, which everyone widely recognizes as an organization that has transformed itself into a model social business. The brand famously initiated social listening and customer care in response to the firestorm created by Jeff Jarvis's June 2005 blog post "Dell Lies. Dell Sucks." Since then, the organization has been a model of social innovation--Dell made millions with its @DellOutlet Twitter handle; it launched Dell Ideastorm to co-innovate with consumers; it deployed a social listening command center; and it used social listening to recognize and immediately resolve concerns with XPS 13 pricingIn 2011 when Forrester went looking for the most socially mature "Empowered" companies, Dell was an obvious choice for the short list.

Just one question: Where is the evidence of Dell's success? Since June 2005 when Jarvis posted his criticism of Dell until October 2013 when Dell went private, the company's stock dropped 66%. Over the same period, other tech stocks performed much better: Hewlett Packard stock was even, Cisco was up 17%, IBM up 138% and Apple (one of the least social businesses around) was up 1249%. What, exactly, makes Dell such a worthwhile and repeatable example of social success (other than really, really good PR)?

Many other well-known case studies collapse pretty easily when you dig a little deeper. I was quite critical of Esurance's Super Bowl sweepstakes a few months ago because so many bloggers and journalists went gaga for the big numbers reported by the brand. (Why is it when accountants see suspiciously high numbers, they question them, but when marketers do, they immediately tweet, share and blog about them?) I questioned the business and brand value of the tweets made to enter a social sweepstakes, but some people criticized my analysis by pointing out Esurance boosted its follower count from 10,000 to more than 260,000.

It's worthwhile to look at how the brand is doing almost two months later: The account has since lost half its new followers. Even more damning is this: Esurance started promoting its upcoming sweepstakes via its Twitter account on January 27. In the prior seven days, with just 10,000 followers, the company tweeted 12 times and received an average of 14 replies, 9 retweets and 71 favorites. In the last seven days, despite having 1300% more followers than before their sweepstakes, the account has tweeted 25 times and received an average of 4 replies, 3 retweets and 3 favorites. Following the brand's $5 million program, the Esurance account is getting less engagement. Why? Because those new followers are not prospects, advocates or customers; they were nothing but random people who wanted to win an easy $1.5 million!

It isn't just positive case studies that suffer from problems; some of the most well-known negative case studies do not stand up to evaluation, either. As I have written in the past, some of the most famous "social media crises" left no signs of adverse business impact. United Breaks Guitars has been cited thousands of times as proof of how a social media PR crisis can damage a brand, but if it did, there is little evidence to be found; in the six months following the release of the "United Breaks Guitars" video, the company's stock outperformed competitors Delta and US Airways by more than 150%.

How about the backlash that Chick-fil-A faced after people objected to comments from its CEO about the biblical definition of the family unit?  Consumer use of the chain was up 2.2 percent in the quarter following his comments, market share rose 0.6 percent, and total ad awareness was up 6.5 percent.

Or what about the Bank Transfer Day backlash against Bank of America's debit card fee? While credit unions reported picking up $4.5 billion in new deposits, Bank of America saw average deposit balances rise 500% more--an increase of nearly $25 billion in the same fiscal quarter as Bank Transfer Day. As for BoA's stock, in the six months following Bank of America's disastrous fee announcement, shares rose 51% while the Dow Jones increased just 22%.

United Breaks Guitars, Chick-fil-A and Bank Transfer Day are oft-cited examples of social media crises that damaged companies, but as case studies, they are pretty toothless. When in the midst of a social PR problem, it is easy for bloggers and journalists to attract clicks, make unsubstantiated conclusions and declare the situations to be cautionary case studies for others, but how many of these same authors go back months later and look for the short- or long-term impact?

Good case studies are woven from reality and facts and do not disintegrate the moment you pick at one thread. Demand and question more, and we can begin to separate the wheat from the chaff.

AugieRay1

Augie Ray

Director - Global Voice of Customer Strategy, American Express

My background includes more than 20 years of experience in digital and social media, including time covering social media for Forrester, managing a large and diverse team in a digital agency and leading social business at Fortune 500 financial services firms. I am focused on how social media is changing not just the way we market and commnuicate but the way social and digital behaviors impact relationships, consumer attitudes and behaviors and business processes. Social's largest impact won't be within Marketing but throughout the enterpise as new forms of selecting, using and consuming goods and services develop.

See Full Profile >

Comments

The problem, as you have pointed out, with case studies is that people don't know how to use them correctly, not that they don't have value. If marketers understand how to scale and how to pull out the strategic structure of why the case worked, they can move the pieces across to their own strategic plan, if the pieces fit. In my experience, the problem with our profession is that not enough people understand or use strategy to begin with so they can't replicate the pieces in their own campaigns correctly. We are desperate to use the copy & paste to achieve similar results because we don't know any better.

And just a thought about your comments on reach (retweets and fan numbers). There are places where reach is an important goal, but it can't be the end-all, be-all. If you're shooting for top-of-mind, as Esurance was, increasing fan numbers is a first step. But if you quit there, you fail. Marketing is a process, not a one-time act. Reliance on case studies alone will not produce success. But as an idea generator, I think they are valuable. The key is in learning how to replicate correctly, or whether to replicate at all.