How Big Data is Influencing Big Companies

A recent survey by New Vantage Partners, a data and analytics consulting company, sheds light on what large organizations are actually doing with big data initiatives.

The Big Data Executive Survey 2013: The State of Big Data in the Large Corporate World surveyed nearly 100 senior executives from Fortune 1000 companies. The responses indicated that these companies are increasing their commitments to big data projects. Most executives (70%) are looking to improve their “time to answer” with big data.

New Vantage Partners began its big data survey series last year. In the first survey, 85% of respondents said their organizations were just beginning to explore big data, with only 35% investing $1 million or more. Just a year later, the current survey’s respondents have painted a different picture. Here are highlights of this year’s survey findings:

• 91% have a big data initiative planned or in progress
• 60% have at least one big data initiative completed
• 32% have an initiative in production
• 88% expect to spend > $1 million on big data by 2016
• 50% expect to spend > $10 million on big data by 2016
• 14% expect to spend > $50 million on big data by 2016

The survey results are especially interesting for organizations considering an investment in big data — time, technology or dollars — but worried about a lack of talent to push projects forward. Sixty-eight percent of respondents plan to retrain and upgrade existing staff to handle big data initiatives. Only 30% are actively seeking to recruit data scientists, or have already done so to augment their existing staff, and 19% report they already have expertise in house.

At the same time, organizations are looking at new processes, and in some cases new roles, to ensure adoption and use of big data. Over half of the respondents have established a Big Data Lab or Center of Excellence; 61% have established data governance standards and committees, and 56% have established a big data executive committee with management oversight. Nearly half (48%) indicate that their companies have established or are considering a Chief Data Officer role.

Between 60% and 70% of respondents listed their top areas of investment this year as: development of more sophisticated analytics; more effective integration of existing data sources; and the creation of analytical sandboxes to support data discoveries.