The Mirage of Data

By Gerardo Dada
May 20, 2015

Understand the limitations of a data-driven approach.

Rate

An ongoing stream of new technologies gives businesses access to ever-increasing amounts of data as well as tools to more quickly analyze that data. However, it’s misguided to think that data alone enables us to make the right decisions and bulletproofs our path.

Sixteen years ago, one of the world’s most innovative companies was about to launch a massive project. It had hired three top consulting companies and spent eight figures to interview 200,000 people in 54 cities and an additional 22,000 individuals at 3,000 corporations. All this research and data should have guided the project and ensured success, right?

The company was Motorola and the project was Iridium. It went down in history as one of the most colossal business failures ever, resulting in a $5 billion write-off. Motorola had access to experts, technology and reams of data. But the company failed to ask the right questions and to interpret the data correctly. Specifically, Motorola failed to ask if people would pay for the service at a particular price.

The value of data is not really in the quantity, but in knowing what to ask for and deriving the right insights. One of the better-known big-data technologies helps organizations collect well in excess of 100 gigabytes of data per day. To put this number into context, the entire Encyclopedia Britannica uses less than one gigabyte of data.  How are businesspeople supposed to analyze all this data? 

A few years back, Mohanbir Sawhney, one of the most brilliant marketing academics from the Kellogg School of Management, gave my class a problem to solve. The case study included a couple of data tables and multiple data points: prices, costs, volume, market research, etc. Everyone in class fired up their spreadsheets and came up with a couple conclusions, all wrong. Our mistake was in assuming that because we had a lot of data, the answer must be in data analysis. It turned out the answer was in front of us the entire time. There was no need to analyze the data to find the answer, because the answer was found in basic marketing strategy. The data was a trap.

To avoid ensnaring yourself in a data trap, remember the following seven data hazards.

1. Data only looks at the past. Despite conventional wisdom, data is limited as a predictor of the future. Otherwise, it would be easy to predict stock prices, sports scores or even lottery numbers. Albert Einstein knew the act of observation itself alters reality. By the time you collect and analyze data, the environment will have changed, limiting its usefulness.

2. Data does not tell you why or reflect emotion. It is so easy to confuse information with evidence. Modern management practices focus on quantification of all the elements of a business. We are all human. We make emotional decisions and then justify them rationally. From your morning Starbucks, to the car you drive, to billion-dollar-decisions in corporations, it is all driven by emotions. You cannot fully understand customers through a numerical lens. Data can show us what customers purchased, how they paid and how often they visited. But it cannot tell us why.

3. Data is always biased. Every step in data collection and analysis has the potential to introduce bias. Surveys are biased by the subset of customers you survey and the subset of those who respond. Your transaction data is biased based on your existing customers. It may not accurately represent the entirety of the market because you are blind as to why non-customers fail to buy. Collection can result in a self-fulfilling prophecy. For example, when you display the most popular products on your website, they become even more popular—not because they are the best products, but because you chose to display them. Psychologists tell us we are all victims of confirmation bias, giving more value to the data that confirms our hypothesis and ignoring the data that contradicts it.

4. We often look at the data we can collect, not at the data we need. Not all data is useful. Not all data is interesting, and most is not insightful. We tend to assume the data we have (or what we can collect or observe) must have the answer to our questions. The weight of a television set has nothing at all to do with the clarity of its picture. Even if you measure to a tenth of a gram, this precise data is useless. The number of Twitter followers a person has is probably not a good indicator of actual influence, even if it easy to measure. It takes guts to stop measuring things that are measurable and even more guts to create things that don’t measure well by conventional means.

5. Data makes it easy to confuse correlation with causation. In search of an ROI story, social marketers are quick to point out the higher customer value of those who follow a company on social networks, implying they buy more because they follow the company. However, is it more plausible that they follow the retailer on social networks because they are loyal customers? Without understanding the why and the emotions behind people’s decisions, it is hard to properly interpret data. While your data will show that a customer purchased a large-ticket item and a small-ticket item, it won’t reveal which item the customer came to the store to buy and which item was the impulse buy.

6. There may be more than one explanation. It is common to look for the root cause of a problem, to find the one thing that drives customer buying decisions. The reality is that the way people make decisions is far more complex. Dozens of factors influence a single decision. It is like proposing marriage. Do you know why she said “Yes”? Was it your looks, your money, your smile or your sense of humor? As much as we want a simple answer that clearly points to a single explanation, chances are that events and decisions are the results of more complex, interdependent factors.

7. More data (or big data) is not better data. You can survey thousands of people and ask them who invented the light bulb, which will result in a false sense of security in perfectly inaccurate data. We tend to believe that more data gives us more precision, when it usually has the opposite effect. It is easy to draw a chart with two axes and it is possible to compare data using three dimensions. However, adding a fourth, fifth or sixth dimension makes data visualization an impossible problem in itself. With more information and more data points to correlate, we are forced to ignore or minimize large sets of data, increasing our chances of arriving at the wrong conclusion. 

Page 1 / 2

About the Authors

  • Gerardo A. Dada is the vice president of product marketing and strategy at SolarWinds. A marketing strategist, technology marketer and entrepreneur, Gerardo has worked for companies like Microsoft, Rackspace, Motorola and Bazaarvoice. Read his blog at www.theAdaptiveMarketer.com, follow him on Twitter at @gerarodada or email him at gdada@hotmal.com.




Post a Comment

Comment

Allowed HTML: <b>, <i>, <u>

0 Comments