Spurring Innovation Through Competitions

Even the most successful companies have trouble developing breakthroughs. R&D road maps, as helpful as they can be at accelerating progress in known areas, are not particularly effective at spotting new opportunities outside a company’s experience base. Resource allocation mechanisms tend to be biased in favor of innovations that reinforce existing business models. Executives obsess about tying R&D tightly to production and grounding new ideas in reality. And marketing groups often focus on the needs of current customers instead of identifying new market needs, discovering new solutions or identifying new business models. As one executive told us, “We ruthlessly weed out research that doesn’t fit the existing model — those projects last only six months inside our labs. The immune system of the core business is so strong.”

Traditional approaches to generating new ideas — most notably large corporate R&D labs staffed with world-class talent — are expensive and often produce disappointing results. In response, companies are searching for better ways to identify and exploit novel solutions. Increasingly, they are discovering that many of the very best ideas lie outside their organizations, in an ecosystem of potential innovators who possess wide-ranging skills and knowledge. To discover and attract these contributors, organizations are launching competitions and offering prizes.

Throughout history, competition has spurred artists, athletes, scientists and engineers to strive for new heights. As far back as 1714, the British government offered a prize for the accurate measurement of longitude. In 1795, Napoleon offered a 12,000-franc prize to drive innovation in food preservation, spurring a French brewer and confectioner named Nicholas Appert to develop an effective canning process to avoid spoilage. Recently, a variety of organizations have embraced competitions in their efforts to find solutions to challenges as diverse as providing space travel for tourists and predicting patterns of hospital admissions. (See “About the Research.”)

Innovation breakthroughs require companies to explore fundamentally new ideas and opportunities outside the realm of past experiences — in effect, “thinking outside the box.” This involves identifying many opportunities before evaluating them. Competitions help to generate breakthroughs by stimulating diversity in a number of areas: among the individuals that compete, in the organizational forms that contestants adopt, and in the solutions that those organizations develop. Rather than sourcing one solution to a problem, competitions generate many of them. And while some of the proposed solutions won’t outperform the status quo (or the efforts of a highly focused internal team), it only takes one promising “outlier” to open up a new direction. While traditional methods for managing innovation focus on improving the mean, competitions maximize variance. (See “The Power of Diversity.”) Unlike contracting for R&D or writing grants, which require organizations to select the best approaches in advance and invest in only a small number of providers, competitions leverage the entire ecosystem of potential innovators, with the sponsoring organization paying only for the winning solution.

Consider what Netflix, the on-demand video and DVD rental company, did when it wanted to improve its method of recommending movies to customers. Instead of just investing large amounts of time and effort to revamp its algorithm, Netflix set up a $1 million prize to encourage outside teams to improve upon its own recommendation capability by 10%. To help teams develop ideas, Netflix posted sample data online. When teams submitted solutions, Netflix ran them against separate data sets to obtain the final speed and accuracy score. Among the participants were graduate students from China, researchers at Bell Labs and retired management consultants. Between October 2006 and July 2009, when the winning solution was submitted, the company received more than 44,000 entries from more than 5,000 teams. During the first 33 weeks of the competition, more than two thirds of the submissions performed worse than Netflix’s own algorithm. But to Netflix, that didn’t matter. The top 90 or so entries beat the benchmark by 5%, and the best outperformed the benchmark by 7%. What’s more, the entry that eventually won the competition beat the benchmark by more than 10%. Ultimately, Netflix received a nonexclusive license to apply the ideas embodied in the winning entry.

Netflix leveraged competition to find new solutions to a well-defined problem. Other companies are using similar methods to identify new problems and customer needs that can be solved with existing technologies and capabilities. However, competitions can have drawbacks, ranging from the potential duplication of resources to questions about the ownership of the intellectual property. Therefore it’s critical that companies have a clear understanding of the trade-offs before they begin. Managers should understand why competitions can be effective, where and when to consider using them and how best to implement them.

Why Competitions Can Be Effective

Competitions generate diversity in three critical inputs to the innovation process: motivations, participants and organizations. This diversity, in turn, generates a wider variety and greater number of solutions to any given problem.

Diverse Motivations

Economists have a hard time explaining why competitions can be so effective. From an economic standpoint, participants often behave irrationally, devoting too much time and effort to the challenge relative to the expected monetary rewards from competing. In many challenges, competitors in their aggregate (and sometimes individually) spend far more money than the competition prize purse. For example, the 26 competitors in the Ansari X Prize, which awarded $10 million to the first team to launch a three-person spacecraft to an altitude of 100 kilometers twice in the span of two weeks, collectively spent more than $100 million in their efforts to win the prize. While the commercial opportunities available to entrants may have influenced their spending, the same commercial opportunities existed before the competition. And yet more than half of the entering teams didn’t exist prior to the announcement. Why did the competition create so much interest? While our work suggests competitors systematically overestimate their chances of winning, this does not fully explain the over-allocation of effort in relation to expected returns.

Understanding why competitions work requires recognizing a more diverse range of incentives and the ways competitions provide such incentives to spark outsized efforts. Nonfinancial motivations include the thrill of competing, the love of a hobby or pastime, the passion for a particular cause and the reputational effects from participating and performing well. A respected prize has a valuable credentialing effect, legitimizing an innovator’s pursuit of a problem, and a well-designed competition can help entrants build skills and expertise through education and mentoring. For many participants, these “softer” motivations outweigh the financial incentives.

The power of diverse motivations was evident in the 2010 Progressive Insurance Automotive X Prize (PIAXP), a $10 million challenge to develop a vehicle with breakthrough efficiency (as measured by “miles-per-gallon-equivalent” energy consumption). More than 100 teams registered for the competition, and despite the long odds of winning, more than 40 teams incurred the expense of fielding vehicles. When we surveyed the entrants about their motivations, “winning the prize” was only the fifth highest ranked response; more important was their desire to “gain publicity,” “enhance their reputation” or “address environmental concerns.”1

Similar results were found in a study of prizes awarded by the Royal Agricultural Society of England (RASE) between 1839 and 1939 for developments in farming methods and machinery.2 While larger purses attracted more entrants, the largest increase in participation was associated with competitions for the RASE “gold medal” — a prestigious award bestowed on only 13 occasions during the century. Furthermore, these gold medal competitions spurred five times more patent activity than doubling the prize purse. The intrinsic motivation of competing for a gold medal not only increased the number of participants, it also led to significantly greater novelty in the outcomes from their efforts, compared with a purely financial incentive.

Diverse Participants

Different incentives attract different types of participants, not all of whom might consider devoting their skills and attention to another challenge. This dynamic is powerful because in many situations it is impossible to predict who will have the best ideas, or what combination of skills will best solve a problem. For example, in the Netflix competition, one of the highest-performing entries didn’t come from graduate students or computer scientists but from a 48-year-old retired management consultant living in London named Gavin Potter; Potter was able to combine his undergraduate knowledge of psychology with mathematics coaching by his high-school-age daughter.3

Similar patterns have emerged in competitions run by InnoCentive, a company based in Waltham, Massachusetts, whose online platform brings together organizations seeking solutions to problems and potential solvers from all over the world, awarding prizes for the best solutions. Research shows that in many cases the winners of InnoCentive challenges come from outside the field of expertise in which a solution is expected to reside — underscoring the link between diverse participants and a greater range of outcomes.4 Sometimes the best solution already exists, but in a different context that happens to share similar challenges. In other cases, a person or team will approach the problem from an unforeseen perspective, opening up new possibilities not previously considered. Ultimately, searches defined too narrowly within a “traditional” field of expertise (for example, chemistry) can exclude valuable insights that come from exploring different perspectives and viewpoints (for example, biology or physics).

Diverse Organizations

Competitions also encourage different types of organizations to work on a problem. Traditional R&D teams are designed to meet the narrow, predefined goals of a company and are bound by the organization’s usual approaches to problem solving. Attempts to break out of these organizational limits typically fall short and often create their own problems in terms of design and governance. Competitions, in contrast, bring into the game a wide variety of organizations pursuing a range of different goals: from public companies, to startups, to universities, to high schools, to groups of friends who share a common passion. Rather than being constrained by a rigid set of rules and norms, individuals or groups can design an organization that best fits their view of the problem and the solutions they are trying to develop. Competitions provide one mechanism to harness this organizational diversity.

The $30 million Google Lunar X Prize, which challenges teams to land a rover on the surface of the moon and to broadcast high-definition video back to the Earth, offers a case in point. Rather than only attracting “the usual suspects” from the aerospace industry, the contest currently includes dozens of competing teams that represent a wide spectrum of contestants. For example, one team, Next Giant Leap, originated as a consortium of smaller aerospace companies, media and marketing organizations, and students and faculty from MIT. Team Frednet is made up of virtual collaborators, mirroring open-source software organizations. And in an industry where export controls and government funding limit the opportunity for international participation, some entrants are leveraging broad global partnerships in pursuit of the prize.

Critically, recent research has shown that these different organizational forms tend to develop different types of solutions, even when trying to solve the same problem.5 In essence, there seems to be a strong link between the design of an organization and the types of solutions it produces. This phenomenon, known as “mirroring,” occurs because an organization’s governance structures, problem-solving routines and communication patterns act to define the space in which it searches for new solutions. Competitions provide a way to capitalize on organizational diversity.

Diverse Solutions

The combination of a broader range of motivations, different types of participants and differing organizational forms expands both the number and diversity of the proposed solutions. It is this diversity that makes competitions so appealing to companies seeking breakthroughs.

The entrants in the PIAXP competition ranged from established companies like Tata Motors to car enthusiasts and high school students from Philadelphia. The entrants took a variety of technical approaches to developing a 100-mile-per-gallon vehicle capable of meeting strict range, safety and emissions standards. Some teams started from scratch while others tried to adapt existing platforms for efficiency. They came up with a range of designs: three- and four-wheeled cars, two- and four-seaters, and in one case an aerodynamic motorcycle equipped with “training wheels” for stability at slow speeds. In addition to submissions based on electric, hybrid and internal combustion engines, one submission proposed a vehicle powered by gas made from wood and paper products. Teams found improvements in fuel economy through innovative power trains, battery management techniques, improved aerodynamics and lighter materials. Significantly, no single innovation stood out as a “magic bullet” solution. Rather, contestants presented a mix of different technologies and approaches demonstrating that improvements in system design and integration were the key to superior performance.

When to Use a Competition

Once managers understand how innovation competitions work, they need to evaluate the extent to which competition can support their innovation objectives. Rather than pursuing “open innovation” competitions just because others are, managers should assess where their organization needs to improve and how a prize might further those objectives.

Discovering innovations can be seen as a search for the high-value peaks in a company’s “innovation landscape.” Each point in the landscape suggests a different possible design based on a combination of inputs resulting in value. Although some innovation landscapes are predictable, others are rugged and uncertain, requiring significant amounts of trial and error to identify the peaks. Traditionally, companies have relied on internal R&D teams to explore the landscape. However, when the terrain is less familiar, many organizations are turning to competitions.

Consider the annual NYC BigApps competition, which New York City Mayor Michael Bloomberg initiated in 2010. The goal was to encourage software developers to use government data to create applications for solving city problems and meeting local needs. Prior to announcing NYC BigApps, city officials didn’t know which problems were worth solving or how best to tackle them. So they decided to have a competition. In the first year, the $20,000 prize generated submissions from more than 80 individuals, businesses and nonprofit organizations. The proposals addressed a variety of services, including restaurant and taxi driver reviews, applications for finding nearby subway stations and tools for evaluating local schools. Using a platform named ChallengePost, software developers were able to showcase their solutions, appeal to investors and link to social media networks. The contest enabled New York City officials to identify a wide range of high-value applications and put them quickly into the hands of potential users.

To map an organization’s innovation landscape, we have found it helpful to use the classic familiarity matrix,6 which examines the nature of the problems companies are trying to solve and the proposed solutions — specifically whether they are familiar or new. (See “Mapping the Innovation Landscape.”) Organizations tend be good at exploring the landscapes of familiar problems and solutions (in other words, those near the “home” quadrant). They need help when developing new solutions to existing problems, new market needs that can be addressed with existing capabilities and new business opportunities created by discovering new solutions to novel problems. These are the areas where competitions have proven most helpful.

As an example of how an organization can generate new solutions to a known problem, consider the Oil Cleanup X Challenge, a $1.4 million prize awarded in 2011 for the most effective product to recover oil from the surface of the sea. The competition for the prize, which was sponsored by philanthropist Wendy Schmidt, drew more than 350 entrants. The winning team, Elastec/American Marine, of Carmi, Illinois, had been working in the oil recovery industry and selling products in this area for many years. In fact, it had already identified several ideas for significantly improving oil-spill cleanup performance, but the ideas had remained on the drawing board until Schmidt announced the prize. Spurred by the challenge, CEO Donnie Wilson assigned ten engineers to flesh out the ideas during an intense 60-day effort. Their breakthrough solution featured rapidly spinning plastic discs with built-in grooves that create a channel for the oil to adhere to. Their product could recover 4,670 gallons of oil per minute from the water’s surface — four times the prior industry standard.

Sometimes, a company can have a wealth of capabilities and possible solutions but have difficulty identifying which problems these could be deployed against, or who the customers might be. Such was the situation at IBM. Historically, the company had a wealth of R&D discoveries, many of them never converted into commercial products. In 2006, management set out to change that with the launch of an “Innovation Jam,” an online conference for 150,000 employees, partners, customers and researchers.7 The organizers shared information on IBM’s emerging technologies during a 72-hour online brainstorming session aimed at capturing ideas for commercialization. Subsequently, a team of managers sifted through the most promising ideas, which were subjected to a second 72-hour “jam” to define actionable investment areas (such as smart healthcare payment systems and the “3-D Internet”). Within the year, then-chairman and CEO Sam Palmisano announced $100 million of investments in ten business areas identified through the Innovation Jam.

Organizations seeking to build brand new businesses around innovative solutions to novel problems have slightly different needs. Here competitions are often used as part of a broader innovation strategy that combines seed capital, early-stage partnerships and other “open innovation” approaches. Competitions, especially those following business-plan-style formats, provide an excellent vehicle for identifying a large number of early-stage opportunities, evaluating them and building communities in emerging areas.

Cisco’s I-Prize competition, which invited global contestants to help the company discover new billion-dollar businesses, provides a powerful example. After launching the competition internally, Cisco opened the I-Prize to external participants in 2007, offering a $250,000 prize for ideas that would help the company leverage its industry positions to build new businesses. Cisco received more than 1,000 submissions. Using a crowdsourcing platform called Brightidea and a team of judges, the company winnowed the entries down to 40 semifinalists, who spent six weeks refining their business plans before presenting them to a panel of senior executives. The winning team, based in Germany and Russia, developed a sensor-enabled smart grid that improves energy efficiency by taking advantage of Cisco’s leadership in internet protocol (IP) technology.

How to Run an Innovation Competition

Tapping into the power of competitions involves more than publicizing a problem and waiting for solutions to flow in. To be effective, competitions must be designed and managed well.8 We have identified five critical design decisions — what we call the “Five Ps”: 1) frame the problem; 2) establish the prize; 3) select the participants; 4) define the process; and 5) build the platform. Considering these steps allows companies to assess the costs and benefits of using prizes.

Step 1: Frame the problem

Depending on the company’s objective, competitions can be configured in different ways. It’s often helpful to carve the problem into “chunks” to reduce the amount of effort required to enter a competition and attract a larger and more diverse set of participants. However, this approach has a downside: Major breakthroughs may not conform to preconceived notions of what the problem is, or how it can be solved. When dramatic performance leaps are required, companies should encourage participants to think as broadly as possible and consider a wide range of solutions.

To encourage development of a more fuel-efficient airplane, for example, a sponsor could divide the problem into distinct areas, such as improving the body aerodynamics and increasing the efficiency of the engines. Recognize, though, that each time you narrow the focus you embed assumptions about the nature of the problem and the expected solution. This may not be desirable if developing radical innovation breakthroughs is the goal.

Step 2: Establish the prize

Competitions with bold and difficult objectives tend to provide larger rewards to attract attention and lure a more diverse participant base. But the opportunity to win a big pot of cash isn’t as important as you might think. In September 2010, fewer than 130 teams registered for a chance to win the $10 million PIAXP Prize for the design of super-efficient vehicles. That same month, more than 500 teams entered a $10,000 contest sponsored by The Economist to find better ways of capturing atmospheric carbon and reducing global warming.

Reward structures should take into account the range of motivations that inspire individuals and teams to compete. In some cases, monetary rewards will be the prime driver. In others, nonfinancial benefits, such as the opportunity to advertise one’s skills or achievements, may be more important. To attract entrants looking for excitement and intellectual challenge, contests can build and nurture a sense of community and support social interactions among competitors. For contestants looking to build their own businesses, competitions can provide opportunities to network with potential investors, partners and customers. Sponsors can also leverage their own strengths to increase the value of participation. Netflix, for example, shared massive data sets with contest participants. By sharing something of value, organizations can amplify the impact of a competition and attract more participants.

Step 3: Select the participants

Some companies open competitions to everyone. Others restrict entry to prequalified participants, such as the employees of a single company, its customers and business partners. While the power of diversity argues for “openness,” there are circumstances where being too open can be problematic. For example, a competition may require sharing details about the company’s intellectual property. A related issue is the time and effort companies must devote to evaluating submissions.9 In circumstances where assessing ideas is difficult or expensive, prequalifying participants may make sense. Similarly, concerns about safety and privacy may argue for a more controlled process.

Step 4: Define the process

One of the most important ways competitions create value is by encouraging collaboration. Time and again, we saw individuals and teams engage in unprompted and unexpected collaborations. Sometimes, these activities occurred in the course of efforts to build teams that possessed the skills and resources to compete. On other occasions, however, they occurred in spite of the competitive dynamics. Competitors often shared a passion for the challenges that they were collectively engaged in, giving them a common purpose. Contest participants therefore seemed more willing to share information, help others and solicit help than they would be in their normal roles as employees competing in a marketplace.

To capitalize on this dynamic, companies should look for ways to harness the power of competition and collaboration. For example, the X Prize Foundation hosted face-to-face meetings for PIAXP entrants to promote team networking and learning. Although there is always a risk that teams may withhold their best ideas while attempting to discern their competitors’ secrets, the culture of a competition typically discourages such actions.

Step 5: Build the platform

Besides prize money, an effective competition requires a range of other investments: in information technology and processes, staffing and judging, and marketing. A fundamental decision for companies considering establishing a prize is whether to build this infrastructure themselves or tap into external assets and third-party expertise.

Over the past decade, several organizations have developed platforms for running competitions. Companies such as InnoCentive, NineSigma in Cleveland and Kaggle in San Francisco offer comprehensive platforms for hosting competitions; other companies such as Idea Crossing in Cleveland and ChallengePost in New York City specialize in software platforms. On a larger scale, organizations such as the X Prize Foundation have developed a repertoire of capabilities for designing, running and judging multiyear challenges.

So when does it make sense for an organization to build its own competition infrastructure? With Netflix, the critical factor was the level of complexity, both in terms of the data the company needed to provide to participants and the systems required to test performance. Releasing the data to a third party raised privacy concerns, quite apart from the difficulty of having an outside group run the tests to select a winner. Similarly, Cisco and other companies that encourage open-ended proposals often prefer using their own infrastructure because it allows them to tailor the contest platform to their specific needs. Ultimately, the more unusual one’s needs and the more diverse the range of solutions, the stronger the argument is for creating your own prize platform.

Assessing the Costs and Benefits

The advantages of using competitions to pursue innovation must be set against the potential costs and risks. There are inherent trade-offs between the power of diversity, on the one hand, and the costs of generating, evaluating and capturing value from diversity, on the other. The costs fall into several categories, each of which must be evaluated before making the decision to run a competition:

Prize Infrastructure: Successful prizes provide something attractive to potential participants — money is not enough. Companies need to provide data or a development infrastructure to allow opportunities to be identified or problems to be solved.

Prize Administration: As the CTO of Cisco’s emerging technologies group has warned, “Anyone attempting to do innovation on the cheap should look elsewhere.”10 As enticing as it may be to get people to work “for free,” such thinking underestimates the cost of the resources you will need to provide; the administration and operations costs often exceed the prize purse.

Prize Adjudication: If it can be costly to test whether a single solution is a good one, imagine the expense of evaluating dozens or even hundreds of entries. Inviting entrants to develop new algorithms makes for popular competitions; they can be evaluated automatically. Car designs can be tested with a race. But evaluating new designs, chemicals and drugs is more challenging — and more expensive.

Disclosure Risk: In describing the challenge you seek to address, you are sharing important information with the world, which might be helpful to competitors. Some prize platforms help you retain anonymity, but not all of them do.

Control: Any venture in open-source or crowdsourced innovation cedes a large measure of control to those doing the innovative work. The traditional precepts of centralized project management and milestone reviews don’t apply in the same way. While the upside can be substantial, the risk factors are outside your control.

 

Competitions can play a role in the innovation portfolio of any company. Some efforts will be successful and others will fail, which is natural given the uncertainty inherent in competitions. That argues for starting small, experimenting with a variety of different approaches to evaluate which are most effective, and learning and adapting as you go. In order to coordinate efforts and derive maximum impact from the various experiences, companies should appoint a single senior executive as the point person — in effect, a “chief competition officer.” That person should ensure that the company rigorously evaluates proposed experiments in terms of the company’s overall innovation objectives and competitive capabilities. Ultimately, decisions such as how much infrastructure to build, the level of funding for competitions (versus more traditional R&D projects) and how best to market and promote competitions should be made by managers overseeing the entire portfolio of activities under the innovation umbrella.

Particularly in lean times, innovation competitions represent a high-leverage tool that taps into powerful motivations to draw out disproportionate efforts from a wide variety of participants. In addition, they can focus contributors on specific aspects of an organization’s innovation challenges. By varying the structure of competitions, companies have the ability to influence the types of innovators they hope to attract, the nature of the solutions submitted and the amount of collaboration that ensues. Therefore, competitions can be a powerful addition to a company’s R&D portfolio.