Duped by Data
There are many small ways that data analytics can lead decision making astray.
Topics
Competing With Data & Analytics
It’s easy to enjoy and celebrate stories of analytics success. But when analytics leads us astray, the results can be ugly and the stories much less fun to relate. We’re left scratching our heads and wondering how things could have gone so, so wrong.
Unfortunately, we have some notable examples of when data either led people to believe in something that was not true or, conversely, led people to doubt something that was true. In the mid 1980s, The Coca-Cola Co. did extensive market research using focus groups and surveys before deciding to reformulate Coke, its main product for more than a century. But the survey data showed a strong market preference for “New Coke” that did not, in reality, exist. The product launch was a bust, and “Classic Coke” — the original formula — returned within a few months’ time (while “New Coke” was discontinued in 2002). More recently, Volkswagen was found to have intentionally skewed emissions data generated during testing of its cars with diesel engines, obscuring the presence of pollution that did, in fact, exist in real-world driving situations.
While these big stories get attention, I suspect there are many examples of small stories about misleading data throughout lots of organizations. Using a sports-management analogy, the movements toward “small ball” are based on the idea that a series of small changes can add up to wins, with data analysis supporting each small decision. But this cuts both ways. If you believe that doing lots of small things correctly can add up to success, then doing lots of small things incorrectly can lead to failure. A number of small deceits by data can add up.
What’s worse is that managers’ work often doesn’t have clear endpoints, like sports games or elections do. As a result, based on data, managers may mistakenly continue to invest resources in activities that they shouldn’t. For example:
- One machine is taken out of service for maintenance unnecessarily while a seemingly functional machine breaks down. While a breakdown is observable, unnecessary downtime is hard to assess.
- Marketing is allocated that targets uninterested people and leaves potentially interested buyers unaware of a choice. Precise attribution of marketing results is notoriously difficult.
- Misallocated costs kill profitable projects when lamprey projects that prey on company resources survive to claim another victim.
Comment (1)
Praveen Kambhampati