Reducing Unwelcome Surprises in Project Management

Why do so many projects fail to meet their goals for time, cost and performance? Regardless of the answer, many project managers and their executive sponsors seem to be surprised when a new project gets off track: “Why didn’t we see that coming?” Even projects that employ sophisticated techniques for risk management can encounter surprising derailments. Those methods, while powerful, can only manage known risks. But projects are new and unique. What about the things that we don’t even know that we don’t know? These “unknown unknowns” — often called “unk-unks” — are lurking in every project, just waiting to emerge, surprise and derail plans. To what extent are they inevitable? What could we do better?

Project knowledge comes from learning about the project — its overall context, its goals and objectives, the process for achieving them, the people, tools and other resources to be deployed, and how all of these affect one another. This learning begins in the planning stages. One might think that planners would consider all of the scenarios, evaluate all of the options and identify all of the risks — but that is seldom the case. Many planners resist wasting resources on planning projects that may never happen. Even after a project gets the green light, a typical attitude of many managers is: “We’re already behind. We know what we need to do. Let’s get started!” As a result, the distinction between what is knowable about a project and what is actually known can be quite large.

Many so-called “unk-unks” aren’t really unk-unks at all. Rather, they are things no one has bothered to find out. Indeed, there are two kinds of unknowns: unknown unknowns (things we don’t know we don’t know) and known unknowns (things we know we don’t know). (See “Converting Knowable Unk-Unks to Known Unknowns.”) Every project has some of both. The techniques of conventional risk management apply only to the known unknowns. Yet some unk-unks are knowable and can be converted to known unknowns through a process of directed recognition.

This article provides an overview of the targets, methods and tools — the where, why and how — of directed recognition. (See “About the Research.”) First, we introduce six project domains in and around a project where uncertainty resides (and where recognition of that uncertainty should occur). Second, we describe six characteristics that increase uncertainty in projects and explain why they make unk-unks more likely. Finally, we present 11 techniques for converting knowable unk-unks into known unknowns. The goal is to reduce the unwelcome surprises in project management.

Where the Unk-Unks Are: Six Project Domains

Projects operate as systems. Project outcomes and performance result not only from individual project elements but also from how the elements work together. Every project has at least five key subsystems,1 which are enmeshed in the project’s broader context or environment. These five subsystems plus the project’s context comprise six important domains, each of which contains both known and unknown unknowns.

Result Subsystem

The desired result of most projects is a product, a service or some other deliverable. Results have many components, all of which must work well together to deliver success. Problems in one area can spill over into other areas, causing a cascade of problems. For example, the HealthCare.gov project at the heart of the Affordable Care Act of 2010 was more than just an e-commerce site selling insurance; it was a system with complex interfaces to other government systems across a wide range of departments. In October 2013, when there were serious issues with the launch, it was evident that the project had run into messy integration problems with its key product.

Process Subsystem

The work required to execute and manage a project is another type of system, one made up of activities, tasks and decisions related by the flow of information, work products and deliverables.2 Efficient and effective processes depend not only on the activity content but also on the relationships among those activities. For example, a lean, value-adding activity could fail to add value if it receives bad inputs (which in turn could impact other activities and cause problems later). Because the network of activity relationships and its implications can be hard to see and manage, the process subsystem is often rife with latent unk-unks.

In March 2008, for example, British Airways and the British Airport Authority suffered a huge embarrassment at London’s Heathrow Airport Terminal 5. After the project’s use of the latest thinking in lean project management had been touted, the opening of BA Terminal 5 turned out to be a debacle, with hundreds of canceled flights and thousands of lost bags. BA lost $32 million, and two senior managers lost their jobs. While the BA project team had focused on the technical side of the project (such as getting the building equipped and testing the building’s services), it neglected operational logistics and staff training. On the opening day, many staff were late for work (they couldn’t find parking) and weren’t able to log into the computer system. BA’s experience underscores the fact that many unk-unks lurk within the complex network of tasks and relationships composing a project.

Organization Subsystem

The people, teams, groups, departments and functions collaborating on a project represent another type of system. In many cases, this system breaks down due to what is often referred to as “poor communication.” However, the solution isn’t for everyone to communicate with everyone else. Rather, it’s necessary to strike a balance between effective information transfer and information overload. When the organization subsystem is suboptimized by miscommunication, a lack of communication or information overload, the risk of unk-unks grows.

Tools Subsystem

To manage activities and transfer information, people in organizations need tools, facilities and equipment. Today, most tools needed for activities such as information exchange, compatibility and service support are software-based. Unfortunately, many software tools are unable to transfer data due to various incompatibilities and organizational decisions. For example, computer-aided design tools can work well in some settings but not in others. When an aerospace design project wanted a certain CAD tool so it could collaborate easily with its partners, the project’s parent organization said it had already standardized around a different brand. Adversities in the tools subsystem can be a significant source of unk-unks for a project.

Goals Subsystem

Most projects have goals for time, cost and performance (functionality, capability provided, quality, scope, etc.). These three areas compete with each other: Improving on functionality, for example, often means increases in cost and/or time. The same often goes for performance: Increasing one capability usually requires a trade-off with another. The goals subsystem influences what is and is not possible, permissible, desirable and effective. As these trade-offs become more pronounced, the possibilities for unwelcome surprises increase.

All five of these project subsystems are related to each other. To accomplish the project’s goals, the organization uses the tools to do the work (execute the process) and produce the desired results. All of these relationships imply no small amount of complexity — which, as we will see, provides a fertile breeding ground for unk-unks.

Context

Every project exists within a larger context. A project may be part of a larger portfolio of projects, or it might have multiple stakeholders who have competing visions and requirements for success. A project’s ideal software tools might be consistent with its parent organization’s standards for multiproject commonality, or they may be completely incompatible. The project context contains a mix of known and unknown unknowns, and it interacts with elements in each of the five subsystems. As managers look to convert unk-unks to known unknowns, they need to consider all six of these domains and their relationships.

Six Factors Driving Uncertainty

Several characteristics of a project’s subsystems and context make surprises more likely. Although unk-unks are by definition specific things we don’t realize we’re missing, it’s possible to look at a project and its context and come to the realization that unk-unks are likely to exist — and why. For example, a large, complex project is more likely to encounter unk-unks than a small, simple project. An organization that is actively looking to uncover unk-unks is more likely to convert them into known unknowns. We have identified six factors — characteristics of a project and its context — that tend to increase the likelihood of unk-unks.3 (See “Factors Contributing to Unknown Unknowns.”) By evaluating a project in terms of these factors, managers can learn why their project might encounter unk-unks — and thereby justify why they should invest in taking a closer look for them.

Complexity

A complex system contains many interacting elements that increase the variety of its possible behaviors and results. The five project subsystems described above each have many elements (components, activities, people, tools and goals) that interact in various ways to generate many kinds of outcomes. All else being equal, the complexity of a project (or a subsystem) increases with the number, variety, internal complexity and lack of robustness of its elements. A project with more tasks, more people and/or more requirements is usually more complex than a project with fewer. When a project’s elements have greater variety (for example, they do three different tasks rather than doing the same task three times, or have a team with representatives from four different functional organizations versus a team with four people from the same function), complexity also increases. The internal complexity of an element (for example, a project composed of five huge tasks versus a project composed of five small ones) also matters. Furthermore, if a project’s elements are robust in the face of change (such as engineering design changes, requirements changes, etc.), then they can act as change absorbers, preventing the propagation of change throughout the system, whereas elements lacking this robustness will amplify complexity.

Other aspects of project complexity depend on the relationships among the project’s elements. As the number, variety, criticality and internal complexity of these relationships increase, so will complexity. For example, a project to develop a product with many interconnected parts (for instance, some requiring close proximity, some needing to transfer energy) is extremely complex — and that is just the product subsystem. Collectively, the subfactors of element and relationship complexity can increase the level of complexity significantly, thereby adding to a project’s likelihood of encountering unk-unks. (See “Situations That Increase the Likelihood of Unknown Unknowns.”)

Complicatedness

Regardless of its complexity, a system may appear more or less complicated depending on one’s point of view. In contrast to complexity, complicatedness is more subjective and observer-dependent. For example, an automobile with automatic transmission is more complex than one with manual transmission; it has more parts and intricate linkages. To drivers, it is less complicated (even though it can be more difficult to fix).4 Similarly, a software application may seem more or less complicated depending on the simplicity and elegance of its user interface, regardless of the complexity of its underlying code. A project’s complicatedness depends on the participants’ abilities to understand and anticipate the project. That depends on subfactors such as the intuitiveness of the project’s structure, organization and behavior; its newness or novelty; how easy it is to find necessary elements and identify cause-and-effect relationships; and the participants’ aptitudes and experiences. The more complicated a project seems to the project manager and other participants, the greater the likelihood that something important will be missed, thus increasing the likelihood of unk-unks.

Dynamism

A project’s dynamism — its volatility or the propensity of its subsystems’ elements and relationships to change — adds to its complexity. A project’s external dynamics are especially likely to affect its goals. Regulatory agencies may impose new rules, customer preferences may change or competitors may alter their strategies. Changes in goals may lead to changes in a project’s results (the product or deliverable) and its means of achieving them. Portions of a project might be outsourced, customers or suppliers might become formal partners and so on. Such changes realign the components and relationships considered to be “part” of the project. And increasing complexity and complicatedness increases the likelihood that a project will encounter unk-unks.

Equivocality

Project work requires a lot of sharing of information. If the information is not crisp and specific, then the people who receive it will be equivocal and won’t be able to make firm decisions. Although imprecise information itself can be a known unknown, equivocality increases both complexity and complicatedness. For example, some projects require a number of participants to attend meetings “just in case” an issue comes up that might affect them. The inability to pin down exactly who needs to be at any particular meeting increases scheduling complexity and the length of meetings and makes for “too many cooks in the kitchen.” In such cases, an attempt to avoid one area of unk-unks ironically increases the likelihood of other types of unk-unks.

Offering incentives for candor can show people that there are advantages to owning up to errors or mistakes in time for management to take action. At the same time, it is imperative to eliminate any perverse incentives that induce people to ignore emerging risks.

Mindlessness

We refer to the perceptive barriers that interfere with the recognition of unk-unks as mindlessness (as opposed to mindfulness). Examples include an overreliance on past experiences and traditions, the inability to detect weak signals and ignoring input that is inconvenient or unappealing. By mindlessly relying on past data, book inventories and existing documentation or components instead of requiring physical verification, managers may be inviting unk-unks. Individual biases and inappropriate filters can keep periphery-dwelling knowledge in the shadows. A project manager’s limited “bandwidth” requires filtering out the “noise” while letting important information through. Unfortunately, filtering is prone to errors, and the information that gets screened out, willfully or not, can be critical. Although it can be tempting to suppress or dismiss negative information while accentuating the positive when promoting a project, that can be a slippery slope. Mindlessness increases a project’s susceptibility to surprising unk-unks.

Project Pathologies

Whereas mindlessness pertains largely to the individuals associated with a project, project pathologies represent structural or behavioral conditions in and around projects as a whole that allow unk-unks to remain hidden. Project pathologies include mismatches among the project subsystems and context (for example, goals for which no organizational unit is responsible), unclear expectations among stakeholders and dysfunctional cultures. A dysfunctional culture can manifest itself in numerous ways: information asymmetries (for instance, some stakeholders have key information about a risk that others lack), shooting messengers, covering up failures, discouraging new ideas and making some topics taboo for discussion. A manager might interpret a lack of active opposition as positive support, but in many organizations people who harbor doubts keep quiet, knowing that opposing views (either negative or positive) simply aren’t welcome. Each of these project pathologies can make unk-unks more likely by decreasing the likelihood of uncovering them before they become unwelcome surprises.

How to Reduce Unk-Unks

Each of the six factors that increase the likelihood of a project encountering unk-unks can affect each of a project’s six domains, yielding 36 places to look more specifically for knowable unk-unks. How should a manager go about looking? What techniques can a manager use to shine a light on these areas? We have identified 11 tools that can help managers with directed recognition: seven are project design approaches and four are behavioral approaches. (See “From Unknown to Known Unknowns.”)

1. Decompose the project. Modeling a project’s subsystems — to understand their structures, how their elements relate to one another and the subfactors of complexity — builds knowledge that helps expose unk-unks. Decomposition should begin with the natural structure of the overall purpose of the project (the “problem”), identifying the subproblems relating to key areas (such as customer need, product functionality and the venture team) and complementing it with experience and experimentation. For example, one company was able to decompose a project5 by:

a) Identifying the problem’s goals, context, activities and cause-effect relationships

b) Breaking the domains into smaller elements — such as product modules, process activities and stakeholders

c) Examining the complexity and uncertainty of each element to identify the major risks (known unknowns) that needed managing and the knowledge gaps that pointed to areas of potential unk-unks

d) Managing the selected pieces of the project in parallel with different project management methods — for example, treating various project threads as “options” and determining further actions contingent on the outcomes.

2. Analyze scenarios. Scenario planning involves constructing several different future outlooks.6 Unlike many approaches to forecasting, it accepts uncertainty, tries to understand it and builds it into the reasoning. Rather than being predictions, scenarios are coherent and credible alternative futures built on dynamic events and conditions that are subject to change. Scenario analysis looks at how indirect threats or situations affect stakeholders, competitors, suppliers and customers,7 and it is particularly suited to uncovering unk-unks in projects.

3. Use checklists. Codified learning from past projects can enlighten future planning. This often shows up in the form of checklists or prompt lists. Of course, providing such tools won’t help if they are viewed as obstacles rather than facilitators of success. Checklists and categories need to be viewed as helpful prompts, not substitutes for thinking. Although some professionals such as doctors have sometimes resisted using checklists, airplane pilots have long known that a good checklist helps smart people free up thinking for higher-level problems.8

4. Scrutinize plans. Project plans are merely a hypothesis for how success will occur. At a minimum, plans should contain information about the expected work (for example, when it should start and finish, projected costs, anticipated results, responsibilities and resource requirements). These expectations need to be scrutinized closely by project participants and other stakeholders. The scrutiny can come in the form of reviews, audits and even formal verifications of how the content was generated.9 Just as reliable products may require some redundancy, project plans may need predefined contingencies. An independent board of overseers composed of experienced experts, empowered to obtain all kinds of project information, can help reduce potential unk-unks lingering from planners’ entrapped mindsets. In a well-known case in 1992, NASA’s Mars Observer was lost due in part to a lack of independent verification and validation.10

Although it can be tempting to suppress or dismiss negative information while accentuating the positive when promoting a project, that can be a slippery slope.

5. Use long interviews. Long interviews with project stakeholders, subject matter experts and other participants can be effective tools for uncovering lurking problems and issues.11 However, interviewers need to be careful not to be too enthusiastic about the projects they’re examining and not asking “yes or no” questions. The best interviews probe deep and wide and ask “out of the box” questions, which can help managers identify latent needs that project stakeholders are unable or unlikely to articulate readily. Consider Silverglide Surgical Technologies, a Boulder, Colorado-based company specializing in nonstick electrosurgical instruments.12 It came up with what it thought was a novel product — a nonstick surgical probe. Although surgeons were intrigued by what the new product could do, they weren’t accustomed to using a probe to operate, so the product bombed. Subsequent studies revealed that had the surgeons been asked, they would have preferred nonstick forceps to a probe. That was a knowable unk-unk.

6. Pick up weak signals. Weak signals often come in subtle forms, such as unexplained behaviors, confusing outcomes or a realization that no one in the organization has a complete understanding of a project. Recognizing and interpreting weak signals requires scanning local and extended networks, mobilizing search parties, testing multiple hypotheses and probing for further clarity.13 It’s also helpful to include tools we have previously discussed, such as long interviews and diverse scenarios.

7. Mine data. When vast amounts of data are available from a plethora of databases, electronic data mining can be a particularly powerful tool for extracting implicit, previously unknown and potentially useful information. By simultaneously reviewing data from multiple projects, data mining could enable project managers to identify the precursors of potential problems. The NASA Engineering and Safety Center (NESC) was established to improve safety by proactively identifying precursors to potential problems hidden in NASA’s diverse databases. The NESC found electronic data mining to be a particularly promising tool for the nontrivial extraction of implicit, previously unknown and potentially useful information toward accomplishment of this goal.14

8. Communicate frequently and effectively. Regularly and systematically reviewing decision-making and communication processes, including the assumptions that are factored into the processes, and seeking to remove information asymmetries, can help to anticipate and uncover unk-unks. The 1998-2004 Ladera Ranch earth-moving project in California, for example, needed to find a way to deal with any unk-unks related to discovery of prehistoric Indian ruins or rare animal or plant species the dig might uncover.15 The project manager and the team met weekly to discuss whether the project or its current plan needed to be revised and how. Effective and frequent communication is essential for project adaptability and agility. However, this doesn’t necessarily mean communicating large volumes of information, which can cause information overload. Rather, the key is knowing how to reach the right people at the right times.

Managers can stress the limits to what can be known about a project, especially at its early stages. They can cultivate a culture of healthy skepticism about projects purporting an absence of risk.

9. Balance local autonomy and central control. Many unk-unks are obscured by the relationship complexity and dynamism of a project — dictated by diverse technologies, geographic sites, interests and external influences. Such confusion makes the project management team vulnerable to unwelcome surprises. Using decentralization of control to grant autonomy to the local nodes of a multinodal project facilitates adaptation and innovation as well as recognition of unk-unks (such as the effect of regulatory changes and customer preferences). Although decentralization helps project managers compensate for their knowledge gaps, it creates challenges for governance. Local nodes are less willing to report problems. To achieve adequate control, project managers may adopt an approach that combines bottom-up empowerment to correct errors with top-down efforts to embed learning across the project.16

10. Incentivize discovery. Some of the most promising ways to identify unk-unks include timely and honest communication of missteps, anomalies and missing competencies. Offering incentives for candor can show people that there are advantages to owning up to errors or mistakes in time for management to take action. At the same time, it is imperative to eliminate any perverse incentives that induce people to ignore emerging risks.17 Among the most common perverse incentives are organizational tendencies to stress short-term over long-term results — a key contributor to the financial crisis of 2008.

11. Cultivate an alert culture. An alert culture is made up of people who understand how unk-unks can derail projects and who strive to illuminate rather than hide potential problems. Managers can cultivate a culture of alertness in several ways. First, they can emphasize systems thinking, which recognizes that deciding what to do in a complex system is not simply a matter of repeating what was successful before. Systems thinking also emphasizes the use of multiple perspectives to reach a decision, does not expect to be completely right and changes course in the face of contrary evidence. Second, managers can stress the limits to what can be known about a project, especially at its early stages. They can cultivate a culture of healthy skepticism about projects purporting an absence of risk. Third, managers can seek to include and build a wide range of experiential expertise — intuitions, subtle understandings and finely honed reflexes gained through years of intimate interaction with a particular natural, social or technological system. Fourth, they can seek to develop the characteristics of a high-reliability organization: preoccupation with failure, reluctance to simplify, sensitivity to operations, commitment to resilience and deference to expertise.18 And fifth, managers can attempt to learn from surprising outcomes. In their eagerness for resolution and clear explanations in reviews, managers should eschew the rhetoric of justification and hold out for the possibility of a deeper understanding of the causes of failure.

Projects are complex and complicated. A project’s desired results, planned process, performing organization, tool suite, goals and context all lend themselves to complexity and complication. What’s more, any of these areas can harbor unk-unks, the undetected problems that are buried in the morass of elements and interactions in and around a project. Some unk-unks are actually knowable, but individuals and organizations acting in mindless or pathological ways will allow the unk-unks to remain hidden, where they can fester into even bigger problems before becoming evident. Fortunately, there are tools and strategies to help managers. The 11 approaches described above give managers a tool kit for directing recognition toward uncovering the knowable unk-unks lurking in projects and converting them to known unknowns. By providing guidance on where and why unk-unks exist in projects and how to recognize their clues, managers can reduce the number and magnitude of unwelcome surprises.