BIG DATA is one of the newest overhyped terms in the ever-evolving world of information technology. It was the focus of the Economist conference I attended in Santa Clara, California last week. Entitled INFORMATION-MAKING SENSE OF THE DELUGE, this was a sparsely-attended conference that attempted to bring some sense into how we manage BIG DATA and how it impacts business, government and our own lives. The conference presented a hodgepodge of speakers, academics and luminaries that attempted to address everything about data, but in my view failed to define what BIG DATA is and how to leverage it in business, government and our personal lives.
Editor's note: Peter Auditore is Head of the Business Influencer Program at SAP, a sponsor of The Social Customer.
What was not surprising about this conference was that the majority of academics at the conference looked like fish out of water. Those of us that have a history in data and analytics know that the business intelligence platform emerged from the technology industry not academia.
Who are the Kings of BIG DATA
- Google is the king of big data in my view. Think about it: They know just nearly everything about you, and what nearly the entire world is searching for and buying.
- MasterCard was also at this conference and touted that they processed 12.7 trillion purchases last year and that they have BIG DATA.
- AT&T and Verizon etc. These guys have huge data and even know where you are.
- Amazon and the retailers have huge data sets on what we buy, but only Amazon it seems knows how to leverage BIG DATA.
- CIA, FBI: These guys are collecting data at breakneck speeds and profiling you and your ecosystem in ways you would not believe.
- Social Security: this may be one of the largest BIG DATA sets in the world. After visiting there in 1998 I was amazed at the robotics.
- Utility Companies have BIG DATA, but they are clueless and don't know how to use it.
- And then there is the newest BIG DATA: Social Media data that enables businesses to understand even more about us. However, they don't get it yet.
- Even more interesting is that the majority of BIG DATA now resides on individual devices.
We have exabytes and exabytes (1 billion gigabytes) of BIG DATA now residing on mainframes, super computers, UNIX servers and even in the cloud, but we can't seem to leverage it. One of the most important points of the discussions during this conference came from Rafal Rohoziniski of SecDev. One of the issues with data analysis, he said, is that "we are really good at looking and analyzing data when we know what we are looking for." He continued: "We are not good when we don't know what we are looking for." This is a great point. Visualizing anomalies or unpredictable trends in data sets is not easy. After many years of reading data, I can tell you that this is right on.
Consumer Analytics
According to MasterCard's Gary Kearns, consumer analytics information is the hottest data on the planet. However, it is incredibly dirty and MC spends millions cleaning it. Demand for information on consumer data purchase behavior is through the roof. MC can tell you amazing things, like what are the top five purchasing days in any given zip code, and even what zip codes are moving rapidly from onsite to online purchasing. He did share with us that mobile phones are the new plastic for 18-24 age groups in the U.S.
The concept of the data architect did not come up at this conference, although I did hear the term "data scientist." There is so much information that the real challenge is contextualizing data, especially real time information, and finding the talent that can read the data. Too much data to make decisions creates a scenario called "bounded rationality" according to Hilbert of USC. I guess this is why none of the economists in the world could see the crash coming they had too much BIG DATAJ
Customer Centric Analysis
Web 2.0 and the Internet have always been about instant gratification. Now we want it faster and more insightful so let's get back to us, the customer and the citizen. One of the more poignant discussions in this conference revolved around Andreas Weigend, formerly at Amazon and now at Stanford. His views on leveraging BIG DATA to create an environment of customer-centricity are what Amazon is all about. Weigend believes that data should be socialized to enable customers to make better decisions and that companies should focus on customer-centric analysis of data. He stated: "The socialization of data sets up a world where people share the data and they expect instant gratification from social media." This is right on and exactly what Yelp, TripAdvisor, Airbnb and other social media sites are providing, because people want to express themselves.
In studying the data from call center interaction, Amazon learned that if a customer complaint was not resolved on the first call, the second call was always a nightmare. So Amazon provided information to call center employees focused on second-call resolution and were able to become more customer-centric in handling issues from a second caller. This is really the next generation in what might be called collaborative data analysis which will be further enhanced through Facebook-like collaborative platforms built into business intelligence systems.
Net/Net
Corporations are struggling to handle BIG DATA. And we know that Government can't handle it at all, except for maybe FBI, CIA, DOD and the Navy. One of the best metaphors of the conference came from Al Di Leonardo, who said: "Managing BIG DATA is like managing a kitchen. The kitchen is the data warehouse, the ingredients are the data sets, the analytical tools are the utensils and the recipes are your analysis models and the chief architect is the cook." Until next time I wish you great selling and marketing in the millennium.