Microsite header filler Microsite header
2018
1

Executive Summary

Register to download the full report

Download the Full Report (PDF)

*Registration Required

Accelerating technological innovation, intensifying competitive pressure, and increasing customer expectations are forcing business leaders to rethink how they use key performance indicators (KPIs) to lead and manage the enterprise.

Based on a global survey of more than 3,200 senior executives and interviews with 18 executives and thought leaders, we find business leaders worldwide are struggling to strike a workable balance between tactical and strategic KPIs; operational and financial KPIs; and KPIs that effectively capture the moment while anticipating the future. This imbalance is a source of measurable dissatisfaction and concern as data for KPI improvements continues to increase. Executives also appear torn between adding more detailed KPIs or lasering in on a smaller, simplified set. While no consensus KPI best practice emerged from the survey, we did find a small slice of companies are exhibiting sophisticated data-driven and analytically innovative approaches to maximizing the impact of their KPIs.

Overall, however, most companies do not deploy KPIs rigorously for review or as drivers of change. In practice, KPIs are regarded as “key” in name only; the most prevalent attitude toward them seems to be one of compliance, not commitment. The responses suggest this perfunctory treatment reflects cultural and organizational inertia, not technical or operational limitations. In terms of perceived effect and influence, our survey finds that, ironically, most organizations are KPI underachievers. They get less value than they say they want.

Rapid advances in machine learning (ML) — that is, a machine’s ability to improve its performance based on previous results1 — are poised to radically influence how executives use KPIs to monitor and spur growth. As next-generation predictive algorithms are incorporated into business process planning and design, they seem destined to inspire next-generation digital dashboards. KPIs will consequently offer predictive and prescriptive indicators, not just rearview-mirror reviews. Data-driven companies that leverage these advances by reconceiving their KPIs will enjoy distinct competitive advantages.

These trends — individually and collectively — have particular relevance to chief marketing officers (CMOs) and other marketing executives. These leaders increasingly find themselves accountable for growth-oriented objectives. Accordingly, these executives are exploring new and novel KPIs for assessing growth. More traditional marketing performance measures, such as campaign effectiveness or click-through conversions, are giving way to more customer experience- and advocacy-oriented metrics.

Our 2018 Strategic Measurement Global Executive Study and Research Report, covering organizations in a range of industries and geographies, reveals six behaviors common to advanced users of KPIs:

  1. Use KPIs to lead, as well as manage, the enterprise.
  2. Develop an integrated view of the customer.
  3. See KPIs as data sets for machine learning.
  4. Drill down into KPI components.
  5. Share trusted KPI data.
  6. Aim for KPI parsimony.

This research report, conducted by MIT Sloan Management Review and Google, offers what we believe to be the first cross-industry study of the use of KPIs in the digital era. We explicitly chart executive views on KPI effectiveness; highlight the challenges and opportunities our research uncovered; discuss emerging behaviors by advanced users of KPIs; and recommend actionable next steps executives can take to adapt KPIs in order to thrive in today’s digital landscape.

2

Introduction: Slack Finds Its North Star

The workplace collaboration app Slack launched in 2014. Four years later, a Fast Company magazine profile of the company ventured that it had rapidly become the “de facto tool for workplace communication — a virtual mash-up of the conference room and watercooler, with a pinch of corner office chitchat.”2 Slack operates on a freemium business model, one it has shrewdly exploited by making its free product feel indispensable in short order. The system now has 8 million active users, with 70,000 teams paying for its premium features. Slack is relatively easy to use and makes electronic workplace communications simple to view, organize, and search. By doing so, the product inspires loyalty and motivates users to become paying customers.

Given that Slack is a born-digital upstart, it’s surprising to hear Kelly Watkins, the company’s vice president of global marketing, say that the marketing organization she inherited was structured in a traditional manner. Teams were arranged along functional lines — product marketing, events marketing, brand advertising, etc. — and working with their heads down in isolated silos. That is no longer the case: Watkins today runs an integrated and well-aligned marketing organization focused, in her words, on being “a strategic driver of growth for the business.”

In large measure, that growth can be attributed to an effective partnership between the marketing, product, and sales organizations. The marketing team’s efforts are now focused on ensuring that the sales team has access to a steady stream of companies that are using the free version of Slack and are strong candidates for an upgrade to the commercial offering.

But a broader change — one that’s cultural, organizational, and operational — has been effected through a small number of forward-looking, growth-oriented KPIs that align with the company’s overall strategy. “We’ve worked hard to establish the most top-line and most important KPIs. These are ones that we’re constantly looking at,” Watkins says. “We’re being intentional in terms of saying how many things can we have at that top level and how many things can we use to establish vision and help people understand what matters most.” These KPIs identify, for all employees, the company’s North Star — that is, the top three metrics that together reflect the company’s vision.

Slack’s three top-line KPIs — which Watkins keeps an eye on daily and dives into more deeply on a weekly basis — center on increasing general awareness of the company, accelerating customer growth, and maturing a sales pipeline. These shared KPIs are intended not simply to manage the organization but to give employees the freedom and the agility to make rapid-fire judgments. “If leaders can invest in establishing very specific and concrete KPIs that people can orient their work around,” Watkins says, “the likelihood that a team or an individual can go faster and make the right decisions in the moment, aligned with the outcomes that the business wants to see, is increased.”

The company has a deluge of data at its disposal, of course. Slack is actively exploring machine-learning solutions to improve the organization’s processes. Watkins says Slack is working toward using ML to enable employees to better focus on strategic issues while avoiding the risk of “only paying attention to the data.” There is, after all, “an element of craft to what we do,” she says.

Slack’s use of KPIs reflects several key findings from MIT Sloan Management Review and Google’s first global executive study of key performance indicators. These findings pertain as much to B2B and born-digital companies, like Slack, as to B2C and older companies that are exploring new ways to use KPIs to drive growth in today’s fast-changing, data-driven world.

In the following chapters, we explain the origins of modern KPIs, identify the characteristics of advanced users of KPIs, and offer several takeaways for executives, including what leaders should expect from KPIs as they evolve from compliance-oriented metrics to data-driven growth measures.

3

KPIs and Their Role in Today’s Enterprise

KPIs traditionally have had a retrospective bias by measuring past costs, revenues, and profits but offering little insight into how an organization was likely to perform in the future. Robert Kaplan and David Norton’s balanced scorecard framework, introduced in 1992, revolutionized how businesses connected KPIs to the company’s broader vision.3 The balanced scorecard, incorporating financial and nonfinancial measures in order to guide tactics and strategy in the short and long terms, powerfully influenced a generation of C-suites worldwide. More recently, the “objectives and key results” (OKRs) framework, conceived by Intel’s Andrew Grove and popularized by venture capitalist John Doerr, has proven popular with tech companies to establish, communicate, and track organization goals.

Regardless of the indicators’ historical provenance, the overwhelming majority of executives we surveyed — senior executives in their companies — report using some version of KPIs to measure organizational competitiveness, effectiveness, and success. (In our survey, we defined key performance indicators as “the quantifiable measures an organization uses to determine how well it meets its declared operational and strategic goals.”) Surprisingly, however, a considerable portion does not significantly rely on KPIs to lead their people and processes. Nearly 30% of respondents say their organization’s KPIs only somewhat, minimally, or not at all (!) drive how they lead or manage their people and processes. A larger portion, more than 40%, say their organization’s KPIs are moderately influential. (See Figure 1.)

These findings suggest that KPIs do not enjoy special status as either enablers or drivers of change in many companies and strongly imply there’s no best practice around their use. It is fair to ask — given the absence of rigor and consistency — whether some executives are largely using KPIs in a perfunctory way. That is, are KPIs more about “tick-box” compliance than value-added insight?

More Than Performance Management Tools

In an interview conducted for this report, Kaplan, a senior fellow and Marvin Bower Professor of Leadership Development, Emeritus, at Harvard Business School, recalls that the CEOs of two of the first companies he and Norton worked with on implementing the balanced scorecard framework — then known as Chemical Banking and Mobil — were former officers in the U.S. Marines. Curious about this coincidence, Kaplan asked them whether they saw a connection between their military backgrounds and their willingness to adopt the balanced scorecard. They responded similarly, explaining, in Kaplan’s words, that as officers, “before we send any soldier into battle, every soldier has to understand mission and objective.” The two men carried that military sensibility to the corporate world. As CEOs, Kaplan says, “they felt their No. 1 job was to ensure that every single employee understood the company’s mission and objectives. And they said, ‘We have found the balanced scorecard to be the best tool we’ve ever seen for this task.’”

Among executives surveyed, however, only 26% agree that their functional KPIs are aligned with the organization’s strategic objectives to a great extent. That percentage is surprisingly low and implies a disconnect between functional and strategic goals. One reason is that only 27% of respondents agree that their organization is mostly or predominantly data-driven in its decision-making. (See Figure 3.) That is, the majority of organizations do not see or describe themselves as “data-driven.” This discrepancy inherently undermines the value and potential of KPIs.

After all, KPIs are most effective when used not as ordinary metrics but as the capital-K key metrics that guide organizational decision-making. Strategic alignment via KPIs is more effective (and more likely) with a decision-making culture that relies on strong data capabilities (i.e., data quality, information management, and analytics).

This is particularly true for marketing and sales organizations. As more companies intensify efforts to engage customers beyond the sales funnel, some are refashioning their KPIs to take advantage of more varied data sets and analytic tools.

Increasingly Focused on the Customer Experience

In 1960, economist Theodore Levitt famously argued that many companies were prone to “marketing myopia”: They concerned themselves with producing goods and services at the expense of understanding customers’ wants and needs.4 While the world has been utterly transformed in the decades since Levitt coined the phrase, we again see renewed emphasis on customer focus. This vision is sharpened both by enormous volumes of data enabling companies to learn more about their users and by digital markets that make turning customers into brand advocates and evangelists a pragmatic possibility.

Adidas America is a case in point: Simon Atkins, the footwear and clothing company’s North America brand director, recalls that its KPIs focused exclusively on traditional financial metrics until the company committed to “reengineering our brand with the customer at the very heart.” This reengineering required not just fundamental process change but a profound rethinking of the company’s culture. Adidas struggled to stay relevant to its increasingly digital and mobile consumers. Now, Atkins says, “KPIs that articulate our progress in the mind’s eye of our consumer have an elevated role within our organization.” These include a KPI around Net Promoter Score (NPS), which Atkins allows is “slightly more elusive than sales, profitability, market share.”

Adidas is hardly unique in expanding attention to the customer experience. As Figure 2 demonstrates, when asked the free-response question “Excluding gross revenue, what are your three most important KPIs?” more than a third of our survey respondents mentioned customer-focused KPIs.

In addition, nearly 70% of respondents report that their organization currently has functional — largely tactical — KPIs for customer segmentation. Many others report that they plan to elevate the priority of other customer-focused KPIs in 2018. Functional KPIs highlighting customer lifetime value and brand equity also ranked as high priorities. This survey captures what appears to be a growing recognition that KPIs must begin aligning internal processes with external customer behaviors.

Accompanying this emphasis on customers is a shift toward measures beyond the traditional sales funnel (i.e., metrics earlier in the sales process and after purchase). Many companies are seeking to understand customers in more holistic ways. Sixty-three percent of respondents say they are now using KPIs to develop a single, integrated view of the customer. Tactical KPIs are being combined into more strategic aggregates. Several interviewees explicitly discussed their efforts to understand “the customer journey” — as opposed to the sales funnel — which encompasses the many touch points through which customers interact with brands today.

Sixty-three percent of respondents say they are now using KPIs to develop a single, integrated view of the customer.

KPIs that focus on customers beyond the sales funnel encourage organizational realignment around sharing, coordination, and collaboration. KPIs that are too narrow, too retrospective, or too disconnected from the larger strategy will undermine efforts to achieve customer-related goals. Consequently, the marketing function and the role of the chief marketing officer may take on more growth-oriented, predictive objectives. This shift would have serious implications for working relationships among functions. CFOs, for example, might work more proactively with CMOs and chief revenue officers to establish more growth-oriented KPIs, and chief commercial officers and chief revenue officers could subsume the CMO function. Data governance might become an innovation enabler rather than a compliance activity.

4

The Fundamentals of Advanced KPI Uses

As noted, the survey findings suggest no clear best practice for KPI use and impact. We do see, however, the emergence of pivotal principles when we examine companies that have successfully created alignment around a shared set of KPIs. We created an index built on the concept of KPI alignment — to help determine what leading organizations are doing. This index illuminates and highlights how their actions set them apart from the rest of the pack. (See “The KPI Alignment Index.”)

Simply put, organizations rated as Measurement Leaders treat KPIs differently than those we consider Measurement Capable and Measurement Challenged. Rather than focusing exclusively on how KPIs can help them manage their organization, Measurement Leaders look to KPIs to help them lead — to find new growth opportunities for their company and new ways to motivate and inspire their teams.

Measurement Leaders set themselves apart in the following ways: They use KPIs to lead, as well as manage, the enterprise. They pursue a holistic, integrated view of the customer. They see KPIs as data sets for machine learning. They insist on the ability to digitally drill down to KPI components. They share trusted KPI data. They aim for KPI parsimony — determining which KPIs are most vital and valuable.

Use KPIs to Lead, as Well as Manage, the Enterprise

While management and leadership are in many ways complementary, each has distinct attributes. Managers zero in on shorter-term organizational and problem-solving objectives, while leaders are concerned with innovation and a longer-term vision for the enterprise. Accordingly, there’s a difference between using KPIs to tactically manage a business and using them to lead an organization into the future. Our findings demonstrate that the most sophisticated companies understand that KPIs built around reviewing past performance are less valuable than KPIs that enable them to take advantage of predictive insight. That insight crucially foreshadows the future of KPIs. These leading companies treat their KPIs not simply as “numbers to hit” but as tools of transformation.

Organizations that use KPIs as leadership tools look to leading indicators to frame their strategy. Silvia Lagnado, global CMO for McDonald’s, says the company is frequently using the terms “leading” and “lagging” indicators, and she offers a succinct example for context. McDonald’s has a growth plan with seven drivers, she explains, and each has up to two future-oriented leading indicators and one lagging indicator to measure its success. One of the drivers involves making the fast-food chain a destination. “A leading indicator there is it’s a place I’m happy to bring my children. That’s a brand image attribute,” Lagnado says. “A lagging indicator is guest count growth of trips with families and kids under the age of 13.”

Anand Narasimhan, head of controlling for Asia-Pacific, China, and Japan at Merck, says that his company is also becoming more forward-looking, although he acknowledges that change comes more slowly to the legacy pharmaceutical company than it might to a more agile digital startup. In the last couple of years, however, he says, Merck has evolved from being “a backward-looking organization” to adopting an “integrated business planning framework” that looks “into the rolling 24-month horizon.” “We are always trying to anticipate the future more actively,” he says, “and this allows us to look at more of the lead indicators rather than the lag indicators.” That change has been accelerated, Narasimhan adds, by an abundance of “real-time information and some early wins.”

Use KPIs to Align the Organization

We have seen how Slack’s Watkins created internal organizational alignment around three shared KPIs. Like Watkins, other senior executives we spoke to say they know they cannot succeed if they isolate internal, employee-related KPIs and financial or process KPIs from the customer experiences they seek to create. Measurement Leaders use KPIs to effectively align people and processes to serve the customer and the brand purpose.

Hannah Grove, CMO of the financial services company State Street, says, “The KPI that I pay the most attention to is the voice of the client. What is it that the client needs? What are the insights? Where are we in the journey in terms of client advocacy and loyalty?” Like Adidas, which revamped its culture to keep up with its consumers, marketing leadership for Grove means going beyond customer retention to turning customers into brand advocates. She has KPIs in place (also like Adidas, she uses Net Promoter Score) to keep the company aligned with that vision. She sees this as an evolving “shift to really understanding putting clients at the center of the organization and measuring our efficacy around their advocacy.” Her KPIs are used to help manage State Street’s cultural transformation.

Respondents scoring high on our index are more likely than respondents in the Measurement Challenged category to describe their organization as “mostly data-driven” or “predominantly data-driven.” On the whole, more than one-third of respondents blend intuition and data when making decisions. (See Figure 3.)

If an organization is going to use KPIs to lead, and KPIs are quantifiable measures, then it would by necessity be data-driven in order to lead with KPIs. Marty St. George, executive vice president for commercial and planning at JetBlue Airways, says his “bible, every single day” is a daily revenue report listing how much revenue was collected the day before and how that figure tracks against the airline’s forecast. Platform brands like online hospitality service Airbnb and reservation service OpenTable eschew long-term (measured in years) strategic plans in favor of adaptive, responsive, and anticipatory KPIs that are data-driven and aligned with mission. They prize agility and speed.

Christa Quarles, OpenTable’s CEO, favors flexible, purpose-driven, 90-day targets over longer-term, annual OKRs oriented around strategic planning. While long-term planning is critical for identifying and seizing strategic opportunities, it’s not optimal for measuring and driving teams to outperform in the short run. “When I think about holding someone accountable, their ability to predict what’s going to happen in nine or 12 months is pretty low. So by nature, you’re going to be more conservative, because you just can’t see that far into the future,” she says. “Whereas I’ve found that people can really light the path and set more aggressive targets for 90 days out. Then you get information and signal in that 90 days, and then you set the next target.” For OpenTable, this is as much a cultural as operational decision. The lines between strategic and tactical KPIs are deliberately blurred.

Jonathan Mildenhall, former CMO of Airbnb and now cofounder and CEO of marketing consultancy TwentyFirstCenturyBrand, notes that “there were more people managing data and data science at Airbnb” than there were managing data and data science at one multinational consumer product company where he worked previously. At the significantly smaller Airbnb, he says, “product marketing and product development and engineering drive an awful lot of growth, and they use data in a very, very scientific way.”

Mildenhall continues, “If you’re working at a 20th-century company, then you have to dismantle the received wisdom and try and get data that can give you a real-time assessment of the engagement that your audiences have with the marketing effort and how that engagement drives either conversation or purchase.” He adds that legacy companies are reluctant to do so because “the predominant mindset is one of fear: We must not lose what we already have as we go out to grow.”

Many legacy companies understand this challenge. Lagnado says McDonald’s has “unbelievably powerful data opportunities. If we don’t put an analytics layer on top, we’re missing a big opportunity. It’s my top priority for this year.”

Develop an Integrated View of the Customer

As customer-centric KPIs assume increased importance in a mobile/digital marketplace, their potential is being profoundly redefined. Our survey results demonstrate companies’ clear commitment to creating greater awareness and accountability around customers. Specifically, there’s a desire to develop multidimensional views of customers that draw on diverse functional-unit perspectives. Respondents don’t minimize the importance of more traditional metrics, such as customer lifetime value. But they’re seeking externally focused KPIs that enable them to better segment and engage customers. Such measures complement and build upon more internally focused process KPIs.

As customer-centric KPIs assume increased importance in a mobile/digital marketplace, their potential is being profoundly redefined.

More than 90% of Measurement Leaders affirm that KPIs help their function develop a single, integrated view of their target customer. (See Figure 4.) By stark contrast, that figure drops to 25% for Measurement Challenged organizations. Figure 5 illustrates that across all levels, respondents’ top priority among customer-focused KPIs is to measure customer segmentation (followed by customer lifetime value, brand equity, and churn).

Mukul Deoras, CMO of Colgate-Palmolive, says the household consumer products company is adept at past-performance KPIs, but “we definitely need to worry about KPIs that are concerned with how our consumers are currently interacting with our brands.” Those KPIs will have to account for many elements, he says, including changing consumer sentiments; consumers’ future expectations; how consumers will engage with media in the future; what trends or conversations they’re likely to be a part of; what kinds of conversations the brand can be a part of; and how to keep consumers’ connection to the brand strong. Clearly, an integrated customer view remains an aspiration for many businesses. For example, 41% of survey respondents say they manage digital customers separately from physical customers.

Some marketers, like Slack’s Watkins, generate new insights about their customers by going beyond traditional financial metrics and customer satisfaction KPIs. They’re also motivated by a desire to turn their best customers into advocates and de facto brand champions. We have seen how Atkins brought a customer-focused discipline to Adidas in response to the diminishing importance of the brand’s traditional retail channels. At Airbnb, former executive Mildenhall imposed a similar discipline, which was particularly savvy given that the company has strategic aspirations to turn guests into hosts. “What I brought to a very numerical company, a very data-driven company,” Mildenhall says, “was the data around how the brand was perceived in the hearts and minds of consumers.”

These developments are also changing the traditional role of the CMO. Glenn Thomas, CMO of GE Healthcare, global provider of health care technology and solutions, says the company recently appointed a leader for global customer experience to analytically drive a more holistic view of customer needs. “A lot of the programs associated with customer experience, including commercial engagement, generating insights, and the implementation of digital customer experiences and so on, sit within my shop and will continue to sit within my shop,” Thomas says. “But delivery on customer experience goes beyond commercial. The customer experience is driven by everything from the initial touch that we have from our marketing and sales engagements, to the first time that we install equipment, billing, to a service experience, all the way through our ongoing relationship with them. So it was important to establish a role that stretches across all those parts of the business.” Thomas, too, stresses the importance of customer advocacy as a desired and desirable KPI.

Laura Beaudin, a partner and global marketing lead at management consulting firm Bain & Co., expresses a similar point about the CMO evolution. The CMO role, she says, is “expanding to be responsible for a constant customer conversation across multiple touch points, not just at the beginning or at the top of the funnel anymore. And if you believe that that’s part of what a CMO is going to be responsible for, a metric like NPS becomes more important, because it can give you not only a sense of whether there are certain interactions that matter but also a sense of the health of the customer experience. And that can be a proxy for the brand health overall.” “Brand health,” in fact, has become a holistic KPI in its own right.

KPIs informed by the goal of turning customers into influencers and evangelists are likely to force an organizational, operational, and cultural reckoning in some companies, requiring increased awareness and anticipation of customer needs. Machine learning offers new and novel opportunities for companies to develop a 360-degree view of their customers.

See KPIs as Data Sets for Machine Learning

Executives in most organizations expect that ML/AI technologies will help them achieve enterprise goals. Nearly three-quarters of survey respondents believe that their current functional KPIs could be better achieved with greater investment in automation and ML technologies. Measurement Leaders, however, were almost twice as likely as the Measurement Challenged to affirm that their organization is already investing in automation and machine-learning technologies to drive marketing activities. (See Figure 6.)

Our qualitative data further highlights current aspirations surrounding ML. Sophisticated marketers increasingly use ML not just as a cost-cutting measure but to inform KPI outcomes. Amit Shah, CMO of 1-800-Flowers.com, says, “All of our AI efforts are highlighting for us the central learning we have had, that all of this is helping us learn about our customers, learn about ourselves, and ultimately learn about how we leverage technology, and it has less to do with, ‘What are the workplace savings because we have bots?’” In other words, ML is seen as a platform for value-added augmentation, not just automated cost reduction.

The greater potential of ML is empowering software and systems to learn from data-driven experience. This creates opportunities to use KPIs and their underlying data to “train” ML algorithms. That is, KPIs can be used, individually and collectively, to teach ML systems to improve and optimize their performance. Both survey results and interviews suggest that KPIs are now being thought of as data inputs for ML, not just as analytic outputs for performance review and planning.

Nearly three-quarters of survey respondents believe that their current functional KPIs could be better achieved with greater investment in automation and ML technologies.

Andrew Low Ah Kee, chief revenue officer for GoDaddy, explains how his team uses the web-hosting company’s abundance of data to support KPIs for customer lifetime value. “For any model, you have to set an objective function,” he says. “You can set an objective function for the transactional moment in time, or you can say, ‘Look, I understand this action, this behavior at this point in time. Here’s how it will influence lifetime value.’ Then you can actually train a model to solve and optimize against lifetime value as opposed to solving for transactional period revenue.” GoDaddy explicitly invests in KPIs as data training sets for its smartest ML algorithms.

ML-informed KPIs present comparable opportunities to improve alignment of strategic goals. Thomas says GE Healthcare is on the cusp of such an innovation, which he sees as enabling more predictive KPIs. “As you think about the touch points along the entire customer journey, and not just the commercial journey but the whole life cycle journey,” he says, “part of what we’re moving toward here is much more digitally oriented measurement of those key touch points, and how that transfers into the business in terms of generating insights, identifying ways to optimize our own and our customers’ operations and to create new commercial opportunities.” He describes the mandate he has given to his team of data scientists: “As an example, when you think about marketing optimization to maximize ROI, what I have the team doing is seeing how we get to performance attribution through deduction. So it’s actually boiling out the KPIs from the data rather than setting the KPIs to be measured. We’ll try to derive KPIs from the data and then use that in order to do analysis for targeting purposes, in order to drive commercial impact.”

Despite the promise of ML, Slack’s Watkins describes the company as still in the “early stage” of using it for programmatic marketing efforts. She points to a design challenge: “With highly automated marketing programs, marketers sometimes forget the element of craft in what we do,” she says. “At the end of the day, we are humans attempting to tell stories to other humans to influence the way they see the world. If we’re not staying close to that work and not continuously experimenting and refining new things that can have an impact on how we’re doing automated efforts at scale, we run the risk of getting lazy in our storytelling. We run the risk of only paying attention to the data and not to the quality of the work that comes out of our organization.”

That said, there is a growing sense that companies that do not intelligently invest in ML now will be left behind. Shah captures the urgency: “I think what we will find, five years down the road, is that the people who took the early bets in artificial intelligence actually achieve the learning that cannot be copied,” he says:

And that, to me, is the reason I keep on investing in these tip-of-the-spear outcomes that have very little payoff currently. They are extremely low payoff outcomes, but they are extremely high payoff outcomes in what we are learning and what we’ll keep on learning, and I don’t think you can short-circuit your way the way you can with other channels. I can copy your call script. I can copy your HTML code. I can make your app look like my app. But I think it’s going to be very difficult to overcome one-on-one learning that is potentiated by machine-learning outcomes and algorithms.

Drill Down Into KPI Components

The intricate dynamics that underlie an enterprise’s KPIs can be as consequential as the KPIs themselves: Seemingly simple KPIs, such as click-through rate, often conceal complexities related to time of day, location, and the age or gender of prospective or existing customers, among others.

The intricate dynamics that underlie an enterprise’s KPIs can be as consequential as the KPIs themselves.

Respondents who scored high on our KPI Alignment Index recognize that digitally drilling down to a KPI’s components is critical for effective KPIs. Higher scores corresponded to increased agreement with the statement “I can easily drill down to see the underlying data or analytic components that are aggregated into my KPIs.” Eighty-five percent of Measurement Leaders strongly agreed or agreed with that statement; the percentage was 45 for the Measurement Capable organizations and dropped to below 10 for the Measurement Challenged group. Again, these survey disparities are enormous. (See Figure 7.)

Of the companies whose executives we interviewed, Experian in particular emphasizes segmentation and prospect qualification for its expanding portfolio of services. Jane Yu, senior director of digital analytics and ad operations at the credit reporting agency’s Experian Consumer Services division, is acutely aware that a multiplicity of consumer touch points requires an increased focus on attribution in order to determine which channels have the most impact. Calling conversion rate a “generic and old-school metric,” she says:

There are multiple versions. The conversion in mobile is much shorter than in other channels. Instead of, say, taking four steps for people to convert, on a mobile you only have one or two slots. So, the conversion funnel is quite different. Your starting point and ending point are different as well, so you can’t lump them together. It will mess up your final steps. So, you have to drill down by different segments, whether it’s different channels or different platforms (mobile or web or desktop). And then we even have other acquisition channels, like call centers, which are a totally different conversion path.

In the past, the retrospective/historical bias of KPIs typically meant that determining why specific KPIs had over- or underperformed could take days or even weeks. There was nothing real-time about them. Today, those questions can be answered within moments, if not milliseconds, thanks to the streaming nature of digital data flows and aggregations. Measurement Leaders are six times more likely than the Measurement Challenged to collect all or a significant portion of their KPIs in real time. Legacy organizations with legacy IT systems and legacy financial reporting processes, however, generally lack this capability. Similarly, organizations in the midst of digital transformation initiatives have asymmetric and/or incomplete drill-down capabilities. This limits KPI value and frequently turns KPIs that should be leading indicators into lagging ones.

The ability to swiftly disaggregate and reassemble KPI components creates enormous opportunities for data scientists and their ML algorithms. Those organizations with incentives toward increased automation and ML technologies can more easily drill down to see the underlying data aggregated into their KPIs, and they check their KPI reports more often than organizations without ML incentives. While it is often true that the whole is greater than the sum of its parts, the power to quickly mix, match, analyze, and prioritize those parts can, in dynamic and disruptive markets, make them more valuable than the wholes from which they come.

Share Trusted KPI Data

Our survey results indicate that C-suites and business unit leaders frequently — even regularly — share their KPI data with their functional counterparts. Many executives accept that sharing KPIs, or having shared KPIs, facilitates cross-functional collaboration and helps enable opportunistic efficiencies and outcomes.

Survey respondents’ ability — and desire — to follow KPIs beyond their own sphere of responsibility increases with higher scores on our KPI Alignment Index. While more than half of all respondents monitor or have access to other C-suite or functional leadership KPIs, Measurement Leaders are more than twice as likely as the Measurement Challenged to do so. (See Figure 8.)

Shared KPIs allow different units to collaborate more effectively because managers can see the positive or negative impact of their own KPIs on others. In the Adidas reengineering example, Atkins describes the company’s need for “a much more integrated and fluid approach to winning consumers across all of our organizations.” He says shared KPIs have been integral to Adidas’ transformation:

From an Adidas America point of view, my leadership team has a shared set of KPIs on behalf of the brand horizontally and specifically to their businesses. We have reached out to our sales committee dealing with the wholesale business, to our direct consumer business dealing with brick-and-mortar and e-comm, and have asked and agreed-upon KPIs that are on their docket so that we embed them within our governance to make sure that at the very top and at the very start, we are agreeing on what success looks like.

Of course, greater KPI visibility does not necessarily lead to greater accountability. Our qualitative data suggests that transparent, shareable KPIs can create new dynamics — and, in some cases, conflict — within an organization. As KPIs, enhanced and augmented by ML, become more predictive, enterprises with shared KPIs will be forced to rethink functional boundaries.

Aim for KPI Parsimony

There is no magic number of desirable or effective KPIs for an organization. A larger number of KPIs might help management and leadership pay attention to complex concerns across the business; a smaller number could compel them to bring a laser focus to the company’s most essential strategic goals. Too many KPIs easily become unwieldy, unmanageable, and create unrealistic expectations; too few might result in the neglect of critical business issues. Accelerating access to massive amounts of data — inside the enterprise and out — exacerbates these increasing tensions.

The survey data suggests that most organizations have neither dramatically added to nor reduced the number of KPIs they monitor. Recall that Slack concentrates only on three top-line KPIs. JetBlue’s St. George also looks at three KPIs daily. “If I think about the stuff that I live by, that’s really it,” he says. “It’s the revenues, the operation, and customer satisfaction.” When asked how many of the KPIs they oversee demand most of their attention, a majority of survey respondents chose either the top two or three, or the top 20%.

Our interviews expose a genuine struggle to slow “KPI creep” brought on by data proliferation. “All functions within the modern marketer’s toolset and the stakeholders that the modern marketer interacts with, they are getting more analytical,” says Shah of 1-800-Flowers. “So, the mere presence of KPIs, my bet is that it’s just going to grow more. We are collecting more signals, we are processing more signals, and we have more people who are probably a little bit more literate in reading quantitative signals.”

The biggest challenge that we have today is to sift through tons of meaningless KPIs, focus on really those few, and not get carried away. — Mukul Deoras, chief marketing officer, Colgate-Palmolive

Colgate-Palmolive’s Deoras says there’s a need to winnow down to the few indicators that will help drive growth: “The biggest challenge that we have today is to sift through tons of meaningless KPIs, focus on really those few, and not get carried away. Just because we can measure everything does not mean we need to measure everything. We just need to focus on a few things that are really going to make a big difference to our business.”

GE Healthcare’s Thomas speculates that “we’re moving toward a world where data science and digital — the availability of data, the ability to manipulate and analyze that data — could move us toward individual customer KPIs.”

5

The Future

As both the survey results and interviews suggest, no best practice has emerged around KPI use. Will KPI best practices ever come to light? Do Measurement Leaders offer actionable insights into what KPIs should be? Despite the absence of consensus, certain organizational behaviors and norms positively correlate to constructive KPI outcomes. For instance, clear — and clearly measurable — differences exist between organizations that use KPIs to monitor and assess performance and those that use KPIs to guide and drive performance improvements.

We see data-driven and customer-oriented leaders use KPIs to transform their organization, while those more concerned with “hitting their numbers” remain focused on efficiencies. Who is better positioned to adapt, evolve, and compete? Similarly, more sophisticated managers explicitly use KPIs to promote cross-functional — not just vertical — alignment. For them, KPIs are the means and methods for rigorously defining and measuring the fundamentals that matter.

To be effective, KPIs must truly be “key” performance indicators, not ordinary metrics or measures, that reflect and illuminate the strategic priorities of the enterprise. An organization’s strategy and culture ought to be easily discernible based on its KPI portfolio, and its KPIs should clearly communicate how it tracks value creation and delivers value for its stakeholders — customers, employees, and investors alike.

Our findings suggest actionable next steps for organizations seeking to become Measurement Leaders and to obtain greater value and returns from their KPI investments.

Identify Your Company’s Top Three Enterprise and Top Three Functional KPIs

As we have seen, many organizations struggle to identify the appropriate number of KPIs to prioritize. Profusion has created confusion. Leaders should step up and push their top managers to identify, in writing, the three enterprise KPIs and the three functional KPIs they regard as most important. The number three is a proxy; the point is to encourage focus and prioritization, and to agree on the vital handful of metrics that truly account for business success.

But that’s just a first cut. It is imperative that leadership understands how these KPIs interrelate and align. Is there consensus on how KPIs affirm and support strategy? Are enterprise KPIs primarily financial? Or do customer metrics play an equivalent role? Who owns those metrics and their underlying dynamics? Perhaps even more importantly, how well do the functional and enterprise KPIs align? Are the functional KPIs largely decoupled from the enterprise KPI outcomes? How directly do the functional KPIs contribute to enterprise success?

Create a Process for Ongoing, Enterprise-wide Discussion of KPIs

Enterprise-wide discussion is intended to affirm commitment, not compliance, and to demonstrate that the organization holds itself accountable for getting the maximum possible value from its KPIs. At GoDaddy, for example, each management meeting is explicitly linked to a KPI. Says Low Ah Kee, “Best practice in our business reviews is to start with a scorecard or a dashboard that says, ‘How are you doing on the metrics that matter?’ If a conversation doesn’t begin with that, we haven’t actually aligned on how we’re measuring and defining success.” At 1-800-Flowers and Colgate-Palmolive, KPIs guide discussions of how the organizations will manage innovation and change. At State Street, KPIs, and Net Promoter Scores in particular, are explicitly used to reinforce the company’s cultural and operational focus on clients.

In each case, KPIs are central to leadership conversations around driving organizational behavior and change, not merely assessment tools. If KPIs are not front and center at a management meeting, something may be wrong with the meeting, the management, or the KPIs.

Treat KPIs as a Special Class of Data Asset

Data and analytics are the raw ingredients of KPIs. Surges in data volume and velocity and ongoing algorithmic innovation will require organizations to continually revisit the fundamental elements and attributes of their KPIs. Leaders should be asking: How will access to new data alter expectations around enterprise KPIs? How will mobile customer touch points affect how marketing measures engagement and experience? How will data governance change as real-time access to data and analytics evolves?

KPIs’ special class as a data asset will become even more important as they become an input to ML algorithms and process automation. Call it “the great KPI inversion.” Tomorrow’s top managers may ask whether they are better off using KPIs to inform and manage their people or to train their machines and automate their processes. In either case, data capabilities that support more complex KPIs will likely become a source of competitive advantage in the years to come.

6

Conclusion

KPIs are strategically, culturally, and operationally entwined with how leaders of data-driven organizations define success. This appears especially true for organizations intent on being measurably customer-focused or customer-centric. They are committed to quantification. As companies hone their data and analytics capabilities, KPIs will become even more powerful and persuasive mechanisms for defining economic value and organizational accountability. By contrast, managers who remain ambivalent about data-driven analytics and algorithmic innovation will continue to treat KPIs more as obligations than as opportunities for learning.

A greater volume and variety of KPIs seem inevitable. For instance, the emergence of people analytics and the quantified self (self-tracking data, such as daily steps, caloric intake, or hours of sleep) invites new KPIs for individuals, teams, processes, networks, and customer experience. Consequently, KPI parsimony may become a new leadership art and science, with implications for organizational design and behavior. What transcendent KPIs should business units collaborate and align around? Who owns the customer in these shared KPI contexts?

Embracing next-generation KPIs that emphasize customers, growth, and the future is clearly a challenge for many legacy companies with KPI practices that draw heavily from financial reporting or compliance-oriented traditions. These enterprises may have strategic KPIs, but their metrics cultures tend to be contextually rooted in performance definitions and aspirations of the past. Our survey results suggest that they will face intensifying challenges from growth-oriented organizations using transformation-oriented KPIs. These challenges will likely leave legacy companies vulnerable to Goodhart’s law — the empirical insight that when a measure (in this case, a KPI) becomes a target, it ceases to be a good measure.5 For these companies, KPIs may lead to perverse outcomes.

One of the more intriguing, and potentially disruptive, shifts we’ve observed concerns the ongoing “flip” of KPIs from outputs for humans to inputs for algorithms. KPIs are increasingly used to both train ML algorithms and source or suggest new KPIs. KPIs are not just analytics that inform and influence decision-makers, they also provide data for training ML systems. This provocative inversion has profound implications for how organizations transform themselves.

Tomorrow’s most important KPI arguments and debates will focus on how — and what — performances are truly key to the organization’s future success. What KPI ensembles will inspire — or incite — new value creation initiatives for the enterprise and its clients? For the serious data-driven/analytics-oriented enterprise, KPI definition, development, and deployment will command the lion’s share of leadership time and focus. Leadership will increasingly decide which KPIs will inform human behavior and which will inform machine learning. Those decisions will increasingly determine enterprise success.

i

About the Research

To understand the challenges and opportunities associated with the use of key performance indicators (KPIs) to manage and lead organizations, MIT Sloan Management Review and Think with Google conducted a survey of more than 4,700 business executives, managers, and analysts from organizations around the world in January 2018. Our analysis focused on 3,225 executive-level respondents; more than 1,600 were marketing executives.

The survey captured insights from individuals in 107 countries and 20 industries, representing organizations with a range of annual revenues. The sample was drawn from a number of sources, including MIT Sloan Management Review readers, executive marketing panels, and other interested parties. In addition to conducting the survey, we interviewed business executives from a number of industries and academia to help us recognize the practical issues facing organizations today. Their insights contributed to a richer understanding of the data.

About the Authors:

Michael Schrage is a research fellow at the MIT Sloan School’s Initiative on the Digital Economy, where he does research and advisory work on how digital media transforms agency, human capital, and innovation.

David Kiron is the executive editor of MIT Sloan Management Review, which brings ideas from the world of thinkers to the executives and managers who use them.


Contributors

Carrie Crimins, Masha Fisch, Lauren Rosano, Allison Ryder, Deborah Soule, and Barbara Spindel


Acknowledgments

Simon Atkins, brand director, North America, Adidas America

Laura Beaudin, partner and global marketing lead, Bain & Co.

Mukul Deoras, chief marketing officer, Colgate-Palmolive

John Doerr, chairman, Kleiner Perkins Caufield & Byers

Hannah Grove, chief marketing officer, State Street

Robert S. Kaplan, senior fellow and Marvin Bower Professor of Leadership Development, Emeritus, Harvard Business School

Silvia Lagnado, global chief marketing officer, McDonald’s

Andrew Low Ah Kee, chief revenue officer, GoDaddy

Jonathan Mildenhall, cofounder and CEO, TwentyFirstCenturyBrand

Anand Narasimhan, head of controlling, Asia-Pacific, China, and Japan, Merck,

Christa Quarles, CEO, OpenTable

Amit Shah, chief marketing officer, 1-800-Flowers.com

Marty St. George, executive vice president, commercial and planning, JetBlue Airways

Glenn Thomas, chief marketing officer, GE Healthcare

Kelly Watkins, vice president, global marketing, Slack

Roddy Young, chief marketing officer, Boston Children’s Hospital

Jane Yu, senior director, digital analytics and ad operations, Experian Consumer Services

Tags: , , , , ,

References

1. “Machine Learning,” www.webster-dictionary.org.

2. J.R. Raphael, “30 Incredibly Useful Things You Didn’t Know Slack Could Do,” Fast Company, Feb. 20, 2018, www.fastcompany.com.

3. R.S. Kaplan and D.P. Norton, “The Balanced Scorecard — Measures That Drive Performance,” Harvard Business Review 70, no. 1 (January-February 1992).

4. T. Levitt, “Marketing Myopia,” Harvard Business Review 82, no. 7 (July-August 2004).

5. M. Strathern, “‘Improving Ratings’: Audit in the British University System,” European Review 5, no. 3 (July 1997).