To Succeed With Data Science, First Build the ‘Bridge’

To better align data teams with business operations, a new organizational structure is needed.

Roger W. Hoerl, Diego Kuonen, and Thomas C. Redman Reading Time: 8 min 

Topics


Much has been written about the high failure rate of data science projects. Data science teams often have difficulty moving their insights and algorithms into business processes. At the same time, business teams often can’t articulate the problems they need solved. And they ignore their cultural resistance to data science.

There is a designed-in structural tension between business and data science teams that needs to be recognized and addressed. Structural problems demand structural solutions, and we see a way forward through a data science bridge: an organizational structure and leadership commitment to develop better communication, processes, and trust among all stakeholders.

To understand the challenges, we use the metaphor of a factory (standing in for any sort of business operation) and a data science lab (such as R&D). The “factory” might literally be a factory, but it could also be a business operation that produces decisions on mortgage applications, reads MRIs, or drills oil wells. Factory goals may involve meeting production plans and keeping unit costs low. The watchword for these operations is stability: no major disruptions and few surprises. Factory managers work very hard to establish and maintain stability, so quite naturally they resist anything that threatens it. One important exception is that good factory managers support incremental improvement when it’s quickly followed by stability at a new level, provided that it is driven within the factory. We call these efforts small data projects.

The data science lab, on the other hand, is specifically designed to disrupt the factory. Its job is to find bigger improvements than can be made with small data, change the way decisions are made, and come up with new products that obviate old ones — with a focus on large, complex, unstructured, and data-rich problems. The lab represents the very antithesis of stability — exactly the sorts of things that factory managers hate!

A certain amount of friction between any two departments is to be expected. Factory managers do not appreciate interference from finance when it changes accounting practices, or from HR when it mandates improvements to its employee review system. But these are mere annoyances; they don’t change the ways the factory fundamentally does its core work.

Contrast these annoyances with the full-frontal attack and disruption signaled by the lab.

Topics

About the Authors

Roger W. Hoerl (@rogerhoerl) teaches statistics at Union College in Schenectady, New York. Previously, he led the applied statistics lab at GE Global Research. Diego Kuonen (@diegokuonen) is head of Statoo Consulting, a Switzerland-based consultancy, and professor of data science at the Geneva School of Economics and Management at the University of Geneva. Thomas C. Redman (@thedatadoc1) is president of Data Quality Solutions, a New Jersey-based consultancy, and coauthor of The Real Work of Data Science: Turning Data Into Information, Better Decisions, and Stronger Organizations (Wiley, 2019).

Tags:

More Like This

Add a comment

You must to post a comment.

First time here? Sign up for a free account: Comment on articles and get access to many more articles.

Comments (2)
Manish Thaker
I really appreciate the efforts given on the subject. I would state (in addition to the above points) here that,
organizations are taking 'Data-Science' as a tool/machine with magic for benefit realization. I have seen/experienced the cases where management opens (i.e. provide UID/PW) of 'Data Lake' to the 'Data Scientists' with open ended job to provide/churn out strategy for benefit realization or just benefit. Which in fact is far away then the reality. May be lake of understanding on the other end.
'Data Science' isn't just a process of churning data, but much more than that. One need to understand whole 'DOMAIN'. i.e. business style, market penetrations, environment(business) changes, future aspects/planning/strategy, past History, competitor’s strategy and performance i.e. whole scene to derive probabilistic beneficial strategy. (But mostly the crucial information’s remains in silos i.e. In the EXCEL sheet of key authorities). At many places it's being considered as 'Problem Identifiers' which proves true for small area of research but fails on larger scope. (Obviously it needs more information i.e. sometimes confidential information too.)
It fails, at times, because of 'Over Expectations' rather I will say 'Unrealistic Expectations' with 'False/Less/Bad Inputs'.
And thus the ‘BRIDGE architecture’ is the perfect solution.
Osman AKMAN
Top management in the Organizations must consider on culture of organizations, especially leaders of the departments not only think daily operations but also improvements. Data science team leader and member see themselves as a department leader in order to understand development sides of the department.