Why Managing Data Scientists Is Different
Successfully managing a data science team requires skills and philosophies that are different from those that arise in managing other groups of smart professionals. It’s wise to be aware of the potential organizational frictions and trade-offs that can crop up.
While businesses are hiring more data scientists than ever, many struggle to realize the full organizational and financial benefits from investing in data analytics. This is forcing some managers to think carefully about how units with analytics talents are structured and managed.
How can organizations realize the promise of the evolving disciplines that we broadly call analytics?
Although financial firms were among the first to recruit “quants” to use sophisticated mathematical models and high-powered computing hardware, analytics groups have now taken hold in areas ranging from health care to political campaigns to retailing to sports. Organizations like these can benefit from the insights gained by financial service firms on how best to manage teams doing advanced analytics. It requires skills and philosophies that are different from those that arise in managing other groups of smart professionals.
Rather than just involving oversight and planning, managing a data science research effort tends to be a dynamic and self-correcting process; it is difficult to plan precisely either a project’s timing or final outcomes. For those unused to this type of work, this process can seem quite messy — an unexpected contrast to a field that, from the outside, seems to epitomize the rule of reason and the preeminence of data.
Compounding the friction that this uncertainty generates is the highly technical nature of quantitative research, which can strain relationships between data science teams and other business units. In most organizations, the consumers of data mining or analytic modeling are line managers. However, because many of them aren’t trained in data science, many managers aren’t easily able to evaluate the technical details of a project; as a result they aren’t able to judge the quality of the research — or determine whether a project should take as long as it does. The reverse is true as well: Less experienced data scientists sometimes ignore the rich business experience that line managers could offer them and thus miss out on essential insights that would improve the result or shorten the research process.