Organizational Alignment is Key to Big Data Success

Fortune 500 companies are rushing to make big data investments. Who is leading this charge? What are they doing? What are they trying to avoid? A recent survey of C-level and function heads from Fortune 500 companies offers a unique glimpse into how the captains of industry are thinking about big data and how their companies are changing because of new insights gleaned from big data analyses. Randy Bean, a coauthor of the survey and cofounder of NewVantage Partners, which sponsored the study, sat down with David Kiron, executive editor of MIT Sloan Management Review‘s Big Ideas initiatives, to discuss how top executives at some of the largest companies and organizations in the United States are managing big data.

Tell us about the survey and who you focused on.

We conducted the survey in June and July of 2012. It came out of our executive thought leadership breakfasts and dinners, where we bring together CIOs, CTOs, chief marketing officers and the chiefs of analytics from Fortune 500 companies.

The survey was biased to organizations that have participated in our executive discussions. About 53% were financial services firms, which include banking, asset management firms and major credit card companies. Another 19% were insurance firms. If you put those numbers together, almost three-quarters of survey participants were from some form of financial services firms.

All of the respondents were senior executives in the C-suite or a level or two below. So, this was a unique perspective on what top executives in very large companies are thinking about big data. We had over 50 senior executives complete a very detailed set of 65 questions.

Part of the motivation for doing the survey was that we had heard from many of those surveyed that they wanted to know what their peers were doing in terms of big data. Was big data a real phenomenon across their industry? What kinds of investments were organizations making in big data? What type of applications were they looking at? And did they have people with the right skills to execute big data initiatives or did they need to find people with these skills?

What was the big overall message from the survey?

There were several key themes.

One was the promise of improved data-driven decision making. Everybody agreed on that in principle. All organizations expressed an interest in being able to do that, though what that means for different organizations varies. Some organizations are focused on strengthening their customer relationships; others are focused on managing business and systemic risk.

Eighty-five percent of respondents said they had a big data initiative planned or in progress. Many of these were internally facing, such as improving operational efficiency or the risk environment in the context of Dodd-Frank; [some were] externally facing — how to improve your customer experience or the process of retaining and developing your customer relationships.

Probably the most interesting finding for us was that with these large corporations, it was all about managing the variety of data and not so much about managing the volume. The issues weren’t incorporating larger and larger volumes of information. Most of these firms have been used to dealing with large volumes. That wasn’t the critical issue. It was about integrating information from diverse sources. Across the board, that was really the primary focus of how firms wanted to use big data, and that included incorporating unstructured data.

We were a bit surprised that only 3% said that they cared about social media information. It’s just not in the plans. Some planned to do some very minimal customer sentiment type of analysis, but it’s not top of mind with the large organizations.

Another key finding was that organizational alignment is a very critical factor in ensuring success. There was some division in terms of whether ownership for big data initiatives resided on the business side or the technology side, but there was common agreement that unless the business and technology sides worked together so that there was an understanding of the business objectives and the technology capabilities, a big data initiative would not be as successful as it should be.

The final key finding was about a major roadblock to success, which is having the right people in place with the right skills and the right organizational structure to use big data.

There’s been a lot of talk in the news about the lack of data scientists. Is that really a big obstacle with rolling out big data initiatives?

Well, it’s a popular topic right now, and we do see a lot of organizations saying that they need to go out and hire new people. But we don’t really see a lack of talent as the most critical factor to successful big data initiatives. For us, the critical factors are primarily organizational alignment — getting the business and technology organizations to work together on the common objectives, understanding what the business objectives are and understanding what the technology capabilities are that will support those.

You mentioned that for many companies in your study, the big issue was variety rather than volume. What does “variety” mean in this context?

Well, there are two points here. One is from Tom Davenport, with whom we coauthored a recent piece in MIT Sloan Management Review on how big data is different. Tom has pointed out elsewhere that it’s not really the size and volume of the data. It’s the quality of the problem. The point there is that ultimately, organizations are trying to gain insights. That can be from small subsets of data or very large sets of data. So you don’t want to lose sight of that.

The other point is that most Fortune organizations, particularly in financial services, are focused on integrating a variety of sources of data, including unstructured data, such as legal documentation, trusts, stock filings, corporate actions. Traditionally, they haven’t been able to marry that type of information with structured data, and that’s one of the big areas where organizations perceive that big data can help them. We believe that big data enables big questions.

What is an example of an important business problem that can be solved with a big data analysis of legal documents?

Take corporate actions. These are important to every financial services firm and apply to activities like mergers and acquisitions, financial reporting and any type of activity that’s produced in terms of an announcement. Markets tend to react to the information that’s in these corporate actions. So organizations are trying to correlate corporate actions to actions that take place in the market. In order to do that, they need to take this narrative content and transform it in a way that they can analyze. If they can do that effectively, they will be able to discern patterns that will help them make quantitative decisions based on, essentially, qualitative information.

Who seems to be getting big data and who isn’t? Let’s talk about that in terms of the industries you’ve looked at.

One difference is between how financial services and the government deal with big data, though they both get the importance of it. I had the opportunity earlier this year to visit the Pentagon and meet with the assistant secretary of defense and learn about their Data-to-Decisions program. On one hand, their program is similar to what we see in financial services and private industry around accelerating time-to-answer. How do you translate and analyze vast amounts of data to gain key insights and accelerate that process so you can get from this data to the insights?

But, what was interesting and different about the government and the Defense Department in particular, is that they have information primarily from sensor devices. Think of it as drones and other types of satellite telemetry. They’re trying to think about things like troop movements and activities of that kind, and often the consequences are much higher, at least in human terms. In financial services, the risk is obviously different — incorrect financial reporting, which has different kinds of consequences. In the context of the Department of Defense, the decisions can be life and death in terms of whether to mobilize troops or to launch a missile.

So the human stakes are certainly higher and the volume of information that is available through these sensor devices goes far beyond what the traditional analyst communities deal with. After all, there are thousands of intelligence analysts in numerous government agencies. The degree that technology can enable a more sophisticated, faster ability to sift through this information is critical to their mission.

Whether it be the intelligence community or financial services community, we see this ability to accelerate time-to-answer as being a key metric that quantifies business or intelligence value.

Who is making big investments in big data?

About 85% of respondents said they had big data initiatives planned or in progress. Only 15% of respondents said they had nothing planned right now. In terms of level of investment, 24% said they were investing more than $10 million now, and that was projected to go up to 36 % in the next several years. There were a handful of organizations that indicated they plan to invest more than $100 million within the next three years.

The 15% who aren’t investing indicated that they either have no big data initiatives planned at this time, don’t see an immediate impact and relevance to their business, or believe that, while there’s an opportunity to utilize analytics and big data in a more institutionalized nature within their business and improve their ability to have more data-based decision making, they are not precisely sure where it fits in at this time.

So it varies. For some organizations, particularly in the credit card industry, data is their lifeblood in terms of decision making. They have set up robust discovery environments that have existed for many years. They establish champion-challenger analytic techniques, test-and-learn environments, and recognize data as integral to everything they do. They’re really data-based marketing companies. But that’s in contrast to more traditional types of financial services firms.

Who are investing the most and least in Big Data?

If you speak about financial services broadly, the credit card companies are large investors in big data, and for a couple of credit card companies in particular, data is integral to their whole reason for being, so it’s probably their most important asset — their analytics and data capabilities.

Insurance providers are one of the more interesting and exciting areas because they’ve operated in a very traditional marketplace until now, but they have to adapt to the new regulatory environment. Insurance organizations will have to become much more competitive and cost driven and will have to understand their customers better. In light of the new healthcare laws, they have to tailor insurance programs and care programs to support their customers, and that means they need to have much more robust analytic and data capabilities.

One of the largest insurance companies in America that we deal with recently hired somebody from one of the very large credit companies to bring to bear an analytics capability. They are going to have to fight constantly to make their case why certain decisions should be made based on the new analytics. But that will be an industry that probably transforms more than any other over the next decade in terms of utilizing data to drive their activities.

Where most of the data analysis takes place in insurance firms is in the establishment of what they call informatics functions — the term used within insurance companies for what might be known in other industries as analytics or database marketing. That is going to have to change in order for them to compete effectively under the Patient Protection and Affordable Care Act, otherwise known as Obamacare.

Is big data changing the role of IT in strategic decision making?

That’s a controversial and lively question. My colleague Paul Barth was recently a keynote speaker at the MIT Chief Data Officer Forum, and Paul put forth the hypothesis that the rise of the chief data officer may mitigate the need for chief information officers. He suggested that chief information officers had really become infrastructure officers, and with so much of infrastructure activities migrating to the cloud, the CIO role was becoming less and less relevant to organizations.

So Paul asserted that the CIO role would go away within ten years and the chief data officer would become the dominant role, particularly in terms of guiding an organization’s information strategy. He threw this out to be provocative in an academic discussion setting, and to his surprise most of the CIOs in the audience as well as the chief data officers agreed with him and said that they envisioned this as well.

Two years ago, we really hadn’t heard of anybody in the chief data officer role, but in the past two years virtually all of the major US banks as well as many of the asset management companies and credit card companies have named somebody to the role with the title of chief data officer. What’s interesting to us is that even though people have been named chief data officer, where this role resides in the organization varies. In some instances, it reports up to the CIO or chief administrative officer. In other instances, we see it report up to the CFO or to a line of business executives such as head of strategy or chief operations officer.

We have also seen in one large client organization that the role has been structured so that it reports half to the business side with responsibility for analytics and half to the technology side with responsibility for data governance and data standards.

What key steps should companies think about when launching big data initiatives?

We believe that the most critical thing organizations need to do is have a plan or a roadmap. It’s a commonplace saying, but it’s hard to get to a destination if you don’t have a map to get there. Because big data has become very popular very quickly, part of the reaction has been for executive committees of large organizations to call together the key managers and say, we don’t know what this big data thing is, but we better be attacking it immediately. But that often creates a rush to find people as opposed to thinking through what big data means for their organization.

At one very large financial services firm, we’ve heard that the next executive that uses the word “big data” without a very precise explanation of how it will be used for the organization will be fired. That’s probably an overstatement, but the point is that there’s been so much overuse and misuse of the term that organizations need and want to understand precisely how big data capabilities and big data initiatives will help them solve some of the top five business issues that they’re trying to address right now, be it customer service activities or compliance reporting and systemic risk.

Organizations need to take a step back and think about their key business drivers: What are they? Which ones would benefit from more sophisticated data-driven decision-making, in particular, the ability to iterate through data more rapidly and integrate new sources of data? And then think about how to put in place the processes and make sure they have the organizational alignment and skills to make that happen. Organizations that have a very clear view of what they’re trying to achieve and how they’re going to achieve it are going to have the greatest probabilities of success.

Two other critical issues that were highlighted are the need to establish strong data management processes and practices, and to create a big data environment that can accelerate time-to-answer, which I mentioned before. Let me explain what we mean by that.

We envision two data environments that coexist side by side. One is the traditional production operational environment, which has to be locked down — think of Fort Knox. It’s hard to get things in there, it’s hard to get things out of there, but it’s stable, it’s what’s used for financial reporting, regulatory reporting, customer statements, those types of activity. This is not information that you want to change or trifle with. It has to be bulletproof.

But coexisting with that can be a “discovery” environment where analytics can be used to sift through new data and also to sift through traditional data, and this can be used to discern the new patterns that can later be incorporated within a production environment. So we call this “the new” and “the known.”

The new environment is focused on discovery, and the benefit that big data technology and processes bring is that it makes it possible to “load and go” — which means beginning to access and analyze all of your data without first going through the data engineering process, which is costly and time consuming. The benefit is that organizations can answer critical business questions in seconds rather than days, days rather than weeks, and weeks rather than months.

We had the opportunity to participate in a one-day special invitation big data day hosted by Accel Partners at Stanford University in May, and a number of the big data technology solution providers suggested that organizations could just load all of their data and “go,” but it is not really that simple. It works within the discovery environment because you can rapidly accelerate your ability to iterate through data, but ultimately, you need to move into a traditional schema-based environment. You still need to eventually establish sound data management and data governance, as well as data preparation and data engineering processes and practices.

The “discovery” environment can be an analytics sandbox to rapidly accelerate your ability to discern new patterns and the ability to integrate with a traditional production environment that will feed and enrich one another. We see this as being how organizations will get the greatest value.