Duped by Data

There are many small ways that data analytics can lead decision making astray.

Reading Time: 4 min 

Topics

Competing With Data & Analytics

How does data inform business processes, offerings, and engagement with customers? This research looks at trends in the use of analytics, the evolution of analytics strategy, optimal team composition, and new opportunities for data-driven innovation.
More in this series

It’s easy to enjoy and celebrate stories of analytics success. But when analytics leads us astray, the results can be ugly and the stories much less fun to relate. We’re left scratching our heads and wondering how things could have gone so, so wrong.

Unfortunately, we have some notable examples of when data either led people to believe in something that was not true or, conversely, led people to doubt something that was true. In the mid 1980s, The Coca-Cola Co. did extensive market research using focus groups and surveys before deciding to reformulate Coke, its main product for more than a century. But the survey data showed a strong market preference for “New Coke” that did not, in reality, exist. The product launch was a bust, and “Classic Coke” — the original formula — returned within a few months’ time (while “New Coke” was discontinued in 2002). More recently, Volkswagen was found to have intentionally skewed emissions data generated during testing of its cars with diesel engines, obscuring the presence of pollution that did, in fact, exist in real-world driving situations.

While these big stories get attention, I suspect there are many examples of small stories about misleading data throughout lots of organizations. Using a sports-management analogy, the movements toward “small ball” are based on the idea that a series of small changes can add up to wins, with data analysis supporting each small decision. But this cuts both ways. If you believe that doing lots of small things correctly can add up to success, then doing lots of small things incorrectly can lead to failure. A number of small deceits by data can add up.

What’s worse is that managers’ work often doesn’t have clear endpoints, like sports games or elections do. As a result, based on data, managers may mistakenly continue to invest resources in activities that they shouldn’t. For example:

  • One machine is taken out of service for maintenance unnecessarily while a seemingly functional machine breaks down. While a breakdown is observable, unnecessary downtime is hard to assess.
  • Marketing is allocated that targets uninterested people and leaves potentially interested buyers unaware of a choice. Precise attribution of marketing results is notoriously difficult.
  • Misallocated costs kill profitable projects when lamprey projects that prey on company resources survive to claim another victim.

Topics

Competing With Data & Analytics

How does data inform business processes, offerings, and engagement with customers? This research looks at trends in the use of analytics, the evolution of analytics strategy, optimal team composition, and new opportunities for data-driven innovation.
More in this series

More Like This

Add a comment

You must to post a comment.

First time here? Sign up for a free account: Comment on articles and get access to many more articles.

Comment (1)
Praveen Kambhampati
Just like the outliers can skew the entire data set, a little tweaking of the raw data can bias the dataset into a wrong prediction and misrepresentation. The boundaries and limits for the data representation has to be very explicit. For example, we are all so familiar to the Market research of Nielsens' and the likes who are still based more on manual survey and limited population samples which are extrapolated for larger representations. The larger picture however is devoid of the intricate and necessary local details that authenticate the prediction. We know that the social media data has the capability to reach with more efficacy. However the geotechnical and time to build data could be a constraint in building the datasets from the unstructured dump of Raw data. When done this can can instantly show the impact of local variations in real time, increasing the reliability enormously. Arguably, the market research firms are successful to a smaller percentage to gain access to the larger data enabled facts. There is also a business acumen that is a way doesn't care for the data dependency, and rightly so to an extent. This acumen when enabled with reliable algorithms for prediction can give amazing business results for the organisation deploying the prediction modelling. Algorithms alone can take the organisation southwards for a blind dependency on abstracted facts.