Why It Pays to Be Where the IT Talent Already Is

U.S. businesses are in the midst of a data-driven management revolution. Organizations capture enormous amounts of fine-grained data based on social media activity, radio-frequency identification (RFID) tags, web-browsing patterns, and mobile phone usage. Analysis of this data offers the promise of insights that will revolutionize managerial decision making.

In many cases, however, this type of data analysis has outpaced existing technological capabilities. Many organizations have been left wondering about the economic impact of their big-data technology investments. Are those investments translating into productivity returns?

One particular challenge enterprises face is difficulty in acquiring talent with the technical skills necessary to support big data tools. For instance, Sears Holdings’ early adoption of Hadoop, a framework for distributed processing of large data sets, was initially hampered by a shortage in people skilled in Hadoop as a platform. As demand for big data technologies grows, so do the problems of finding sufficient skills. The result: Talent shortages could limit the rate of productivity growth.

Labor Markets and Technology ROI

To understand the value that organizations are deriving from big data investments, I examined regional differences in the supply of workers and the skills required for new information technologies — especially during new technologies’ early periods, when there are few channels for acquiring these skills. Much of the value of technological investment, after all, is determined by the supply of professionals who can translate technologies into business outcomes. This variation may explain why some organizations unlock value from new IT innovations faster than others. I also examined how labor markets have specifically shaped early returns on investment (ROI) in one big data technology: Hadoop-based systems.

By studying returns on Hadoop investments that are concentrated in select labor markets, such as California’s Silicon Valley, we can observe that the more Hadoop expertise there is in the labor market, the easier it is to hire skilled Hadoop employees. At the time when I collected my data, more than 30% of workers with Hadoop skills were employed in Silicon Valley — a region that accounts for just 4% of total IT employment in the United States. Skills for more mature technologies were much less geographically concentrated.

It turns out that when know-how is scarce, organizations that invest in new IT or R&D derive significant benefits from the related investments of other organizations.