The Mysterious Art and Science of Knowledge-Worker Performance

As far back as 1959, Peter Drucker insisted on the need to pay more attention to knowledge work and the people doing such work. Some 40 years later, perhaps in frustration, he threw down the gauntlet to academics and practitioners alike with the claim that, when it comes to our understanding of knowledge-worker productivity, “we are in the year 2000 roughly where we were in the year 1900 in terms of [understanding how to improve] the productivity of the manual worker.”1 Knowledge work thus far has had no Frederick Taylor or Henry Ford; at best, the subject has been explored by approximations of William Morris and the Italian Futurists (artists who expressed an understanding of industrial developments in the late 19th and early 20th centuries) — such as the architect Frank Gehry, the conceptual artist Jenny Holzer and the design firm DEGW.

Most businesspeople today would agree with Drucker about the importance of knowledge work. They understand that it is at the heart of innovation, which is itself the key to long-term organizational sustainability and growth. It is also a major operational concern: If companies can enhance knowledge-worker productivity in this century anywhere near as much as they did with manual labor over the course of the last one (an increase of roughly 50 times), the payoffs will be astronomical. In the shorter term, recruiting and retaining the best knowledge workers are vital to organizational success. Finally, a focus on knowledge-worker performance is a way of uniting what are often separate tasks, such as strategic planning, organizational design and IT investment.

Given these facts, we have found the problem of knowledge-worker performance, as Robert Oppenheimer once said (for good or ill) about building the atom bomb, one that is too sweet to ignore. And in the spirit of the artists concerned with industrialism a century ago — but with an eye toward more scientific advances — we spent more than a year investigating the subject. (For an overview of our work, see “About the Research.”) Among other things, we learned that sweet problems are not always tractable. Truly sweet problems may require the creation of radically new concepts and tools before they can be solved.

And while we readily admit that we haven’t produced the intellectual equivalent of the atom bomb, we haven’t exactly bombed, either. The first part of this article can be read as a series of dispatches — postcards from the field — that identify the five key issues we saw companies struggling with in their handling of knowledge work. In the second half, we offer a framework that we believe can help organizations think more clearly about how they might go about improving knowledge-worker performance, an objective that should be at the top of most corporate agendas.

First Issue: The determinants of knowledge-worker performance are becoming clear. How to integrate them remains murky.

Following our review of research on knowledge work and the determinants of white-collar productivity, we formulated a simple hypothesis: that three major factors — management and organization, information technology and workplace design — influence the performance of knowledge workers and knowledge-based organizations.2

At one level, our proposition seemed so elementary as to preclude further investigation. After all, research going back decades on all kinds of knowledge work has shown that things like structure, management style and compensation influence performance. And although debates still rage about the impact of IT on productivity in the aggregate, we encountered dozens of studies confirming the idea that new technology helps workers accomplish more complex tasks than they could in the past. The results of research on workplace design were less consistent, but case studies and surveys suggest that physical setting and work-place arrangements have a measurable effect on knowledge work. Indeed, at a conference we organized to engage academic and practitioner experts, there was “violent agreement” about the salience of the three factors.3

What made our hypothesis worthy of further research, however, was the recognition that the three factors have in surprisingly few instances been addressed in an integrated way. Some companies are putting great emphasis on one of the factors, especially IT — witness, for example, Microsoft’s recent creation of a “knowledge-worker solutions” group. And even in the post-dot-bomb era, many companies are toiling to create the functional equivalent of a software Swiss army knife to integrate communication, collaboration, knowledge management, virtual teaming, e-mail and instant messaging. Yet few (if any) have figured out how to get knowledge workers to actually use these tools. Thus far, it all amounts to expensive e-mail.

Work-space manufacturers, architectural firms and design consultants are exceptions that acknowledge the interplay of organizational, technological and physical factors. But we found ourselves wondering if what several managers said to us might indeed be true: that furniture makers did their research solely to sell more chairs. Perhaps that isn’t fair, but the fact remains that the still emerging discipline of knowledge management awaits a model or methodology that can bridge the space between IT, workplace design, business strategy and people management.

Second Issue: Many organizations resist the idea that segmentation of knowledge workers is necessary.

One major obstacle in the way of developing a useful model lies in the generic use of the term “knowledge worker.” We came to this conclusion by reflecting on the comparability of different kinds of work. For example, can an organization apply the same solutions to a hardware-engineering team just because they worked for a sales and marketing team? And can it transfer lessons learned from the organization of customer-service representatives to a group working on the fuzzy front end of chip design?

We were skeptical, but we found that managers weren’t always ready to differentiate their employees. In one phase of our research, we began interviewing senior managers about what we called high-end knowledge work. Our goal was to learn about the treatment of the horses pulling the plow in the knowledge economy: senior professionals, researchers and designers. Like typical researchers, we were unaware of the sensitivities we would arouse when we began to talk about the stratification of knowledge workers. To our surprise, we ran into a variety of objections. Some managers were concerned about perceptions of elitism in their organization, and others felt that segmentation might contradict implicit notions of meritocracy. These feelings were expressed even in organizations that have clear income differentials and substantial rewards and privileges for one form of high-end worker: senior executives.

Despite these objections, we found substantial differences among the types of people who are called knowledge workers and, as we will discuss later, we are advocates of some degree of segmentation. A related point concerns individual choice: A body of research suggests that knowledge workers who have a high degree of autonomy and control over their own work environments find those attributes valuable.4 Companies may not want to recognize this form of self-segmentation explicitly, but they certainly do so implicitly by preserving or designing-in high levels of autonomy and choice.

Third Issue: No one seems to own the problem of knowledge-worker performance.

Whenever we asked managers at the enterprise level who or what function contributes to improving knowledge-worker performance, lots of people raised their hands. When we followed up by asking who is responsible for enabling higher levels of knowledge-worker performance, few hands stayed up.

It’s clear that the managers of knowledge work have a responsibility to optimize work processes, workplace design and technology. Unfortunately, as we’ve been told in numerous companies, line managers often can’t find the time, the resources or the incentive to attend to the issue, because they are expected to focus on current performance. HR and IT professionals sometimes express a keen interest in optimizing knowledge work, but when they aren’t stiff-armed by line managers who fear the disruption of existing work processes, they themselves usually suffer from an incomplete understanding of the processes and thus can’t effectively own the problem.

And when someone does get the inspiration to do something innovative in the way knowledge work is organized, she quickly finds herself crossing paths — if not swords — with several staff groups. The facilities or real estate group, for example, usually has a process and a template for work-space design and gets evaluated on criteria like cost per square foot rather than whether it has created space that contributes to innovation. The HR department might be more interested in tinkering with compensation and benefits — conventional fixes designed to raise productivity. And IT generally doesn’t like the kinds of solutions that knowledge-worker teams relish, such as peer-to-peer networking software (which can threaten system security) and such hardware as personal digital assistants and cell phones (which IT groups don’t support).

We did find a few examples of collaboration among these groups. A team at Cisco Systems, for example, composed of representatives from HR, IT, finance and workplace-resources and overseen by an executive steering committee, meets biweekly to ensure collaboration aimed at improving employee productivity and satisfaction. To that end, the team devised and implemented a new workplace design, as well as guidelines that allow individuals to choose from a variety of work settings to support their needs for collaborative, private or mobile work. Short of a single group volunteering to own the problem in its entirety, collaboration of this sort is the best solution in the short run.

Fourth Issue: Companies are experimenting heavily with workplace redesign, but they aren’t learning very much.

Although companies have experimented with many different approaches in the past couple of decades, we found an astounding lack of knowledge about what actually improves performance. Measures, controls, hypotheses and even the recording of lessons learned are often lacking. Instead, it seems that fad, fashion and faith drive most decisions about new work environments for knowledge workers. The other powerful f-factor is finance. The possibility of finding cost savings is always given paramount consideration.

Consider the many ways companies attempt to increase the likelihood of informal interaction between co-workers or even colleagues from other organizations. We encountered hearths, cappuccino bars, indoor boulevards, creativity rooms, talk plazas and several other designs intended to nourish social life at work. While we strongly support the value of informal relationships — research suggests, as only one benefit, that they are particularly helpful in retaining workers — there is little evidence that any particular design increases informal meetings.5

When we visited companies, we found that social spaces were often empty or being used as conventional conference rooms. Group spaces placed at major floor intersections at a computer company, for example, were quickly filled up with cubicles because no single group owned the spaces. Unless companies push other initiatives to build social relationships — such as frequent networking events or reward and incentive programs designed to foster collaboration — people may be reluctant to be seen in environments that don’t seem like serious work spaces.

At a minimum, senior executives should set an example by using these spaces for their own informal meetings. Within a Corning R&D group, the director makes a point of showing up regularly for the twice-a-day coffee breaks in the “creativity room” she established in order to encourage collaboration between people with different areas of expertise. The director says that many of the group’s best research ideas result from discussions held there.

Open-plan environments are another innovation that a good many managers have championed. And it’s true that over-the-wall chats can help information flow quickly through knowledge-work processes. But we’ve also heard many anecdotes about workers who stay at home to do heads-down work because they can’t concentrate in the office. We also interviewed someone engaged in highly sensitive political risk analysis who feared that his job performance would be severely compromised as soon as the company moved to a completely open floor plan. At Monsanto, one business unit abolished private offices for a time, but senior officers eventually overturned the policy and erected their own private spaces. Many employees are skeptical of open office arrangements and suspect (as do we) that the primary benefit is the lower costs that come from packing more people into cubicles.

Experiments in IT support for knowledge workers have been no more revealing of what works. Formal knowledge-management initiatives, in which corporate portals and repositories have proliferated, have not yet produced substantial changes in the way knowledge workers behave.6 Personal devices such as PDAs are popular but are not well integrated into the corporate information environment. Tools for remote collaboration (from videoconferencing to Webcasts) have had relatively little clear impact thus far.7 And artificial intelligence and expert systems, which offered much promise for knowledge workers in the 1980s and early 1990s, have never taken off (outside of control systems for already heavily automated manufacturing operations) because of difficulties of knowledge representation and system maintenance.8

These experiments with technology have not taken hold in part because they have seldom been accompanied by organizational changes.9 The companies in our study that were most successful in matching IT to their knowledge workers provided extensive training and coaching to help people learn how to select and use technologies that best support a particular job role or task. Organizational efforts of this kind are particularly important for high-end knowledge workers, who have lots of discretion about how they do their work and can usually resist technologies that they don’t like or understand.

Fifth Issue: There is great reluctance to alter knowledge work, especially at the high end.

Knowledge work itself can often seem impervious to change, much less to rationalization. There is, after all, no sure-fire formula for success in creative group processes such as one finds in advertising, strategy making or R&D. Sometimes knowledge workers themselves are the ones resisting change, as they dig in their heels over management efforts to engineer a more productive work environment.

We have observed two dominant approaches to the design and management of knowledge work. The first is a process perspective that works from the assumption that engineering principles can (and should) be applied to all forms of work.

A lot of reengineering in the 1980s and 1990s was aimed at rationalizing lower-level knowledge-work jobs that were relatively structured, linear and repetitive, such as order fulfillment, benefits administration, invoicing and accounts management. Companies and their consultants are now trying to find ways to codify and document repeatable processes in a growing number of less structured tasks, such as product design. The objective is to make them less idiosyncratic, more accessible to a wider array of knowledge workers, and more consistent across families of products (vehicle platforms in the automotive industry, for example).

Partners HealthCare, for example, has done this successfully by linking massive amounts of clinical knowledge, including patient histories and a constantly updated database of clinical information, into real-time IT systems that support doctors’ work processes. When a physician enters an order for a test or a medicine, for example, it is run through a sophisticated logic engine that presents him or her with relevant information and suggestions that the doctor may or may not choose to accept.10

The other dominant view has been called a “practice” perspective. As John Seely Brown and Paul Duguid put it, “Process emphasizes the hierarchical, explicit command-and-control side of organization — the structure that gets things done. By contrast, practice emphasizes the implicit coordination and exploration that produces things to do.”11

The most common approach to high-end knowledge work that we encountered was a form of the practice approach that can be summarized as “hire smart people and leave them alone.” Companies that take this approach pay a lot of attention to recruiting processes and more or less give knowledge workers free rein after they are hired.12 IDEO devotes a lot of effort to ensuring that new product designers fit the culture and mode of work. A new recruit may be interviewed by as many as 20 people before being offered a job. The company also pays much attention to assembling the right teams. As the head of one studio put it, “Our philosophy is that if you put a good project manager together with the right people, the project will be good.”

Other aspects of IDEO’s organizational approach are also representative of the practice approach for high-end knowledge workers.13 The company fosters collaboration by keeping each individual location, or “studio,” small enough — 30 people or less — to ensure strong social relationships. And because high-end knowledge workers tend to dislike hierarchy, IDEO’s structure is relatively flat and the visible perquisites of hierarchy are few.

Yet IDEO also illustrates the need for some process perspective as well, and perhaps points the way toward a compromise between process and practice in knowledge work. The company defines rather firmly the process for brainstorming, which is at the heart of its approach to innovation. Once a clearly defined statement of a problem is posed and a warm-up exercise is completed for those groups that haven’t worked together before, participants in the session begin to generate ideas while obeying strict rules such as no critiquing or debating of ideas. The process is eventually brought to a close after 60 to 90 minutes, when participants vote with sticky notes for the best ideas.

Another blend of process and practice can be found in prototyping, and IDEO puts the prototyping of a new design at the center of the work process. We feel that a continuously iterated prototyping process may be the right mixture of practice and process for many knowledge-worker environments.14 DEGW, an architecture and work-space design firm that is at the cutting edge of knowledge-worker accommodations, offers a good example. The firm works with its clients to create quick approximations to the required solution. DEGW balances the need for structure and creativity with a focus on the evolution of the designed object. Similar approaches are taken in the “extreme programming” approach to software development, in which engineers use rapid prototyping to produce a solution under time pressure.

Toward More Productive Knowledge Work

Although our research findings don’t add up to a single, clear solution to the problem of improving the performance of knowledge workers, we do have some strong convictions that may help companies get started on finding the answers they need.

Our first conviction is that it is a mistake to lump all knowledge workers into one category. They vary in important ways: by the work processes they follow, by status and influence, and by differentiation of work environment.15 The latter is most interesting to us because it offers the best means of observing the interaction of organization, technology and workplace design. Our second conviction is that some degree of choice in how and where to work goes a long way toward making knowledge workers happy. This conviction is backed up by ethnographic research showing that knowledge workers value autonomy and discretion in these areas.16

These two convictions lead us to postulate that if executives can improve the fit between knowledge workers and their environments, it will be possible to start untangling the snarl of conflicting forces that currently impedes performance. We have developed a matrix built on the dimensions of segmentation and choice that we think accounts for much of the variation we found in companies. (See “A Framework of Work-Setting Solutions.”)

Segmenting Work Environments

Companies can segment their work force into many levels, but using a simple “low, medium and high” is best here for simplicity’s sake and captures most of the variation we observed. In general, more segmentation leads to a better fit between workers and their environment.

Low Segmentation

Companies with a low level of segmentation provide one standard work setting for all employees. Employees at SEI Investments, for instance, all work in a large, open environment with identical workstations and technologies. For organizations whose employees engage in relatively homogeneous work activities, standardized work settings can be an efficient way to align the work setting with the needs of the majority. Some organizations may decide to limit or eliminate segmentation for other reasons: when they want to contain costs, underscore a move to a less hierarchical environment, attract and retain workers who value an egalitarian culture, or encourage common cultural values throughout the organization, such as open communication.

This approach is inexpensive and efficient, but it is also most likely to produce a misfit between the needs of individual knowledge workers and their work environments. One manager we spoke with, for example, felt that her company’s move to an open plan would make her less effective by compromising the confidentiality she needed when speaking with clients. Even the executives at SEI, who are happy with the fully open office plan, admit that some potential employees choose not to join the company because of concerns about the work setting.

Moderate Segmentation

Some companies occupy the middle ground. They group their employees into a limited number of categories and define work settings for each. Familiar categorization schemes include differentiating workers according to status, geography or job role. Less familiar but increasingly popular criteria include degree of mobility required by the job, amount of time engaged in teamwork versus independent work, the number of projects undertaken at one time, and the amount and type of communication with others that are needed to do the work properly.

Intel has applied some of these criteria to identify three main types of environment for three types of knowledge worker: “teamers, nomads and sitters.” Merrill Lynch segments its knowledge workers by job role. Brokers and portfolio planners require concentration and are granted private offices as a reward for good performance. Traders, however, require completely open environments, as their success depends on constant communication. IT professionals, who tend to work in teams and perform both collaborative and heads-down work, need yet a third environment: semi-open offices with plenty of team spaces.

High Segmentation

A few of the organizations we studied have highly segmented work settings that are specifically tailored to small groups of people on a one-off basis. Fidelity Investments is one example. It uses a lab facility to help teams and their managers figure out what combination of open and closed space and what type of communication people need to operate most effectively and then designs settings accordingly. Although this approach enables a tighter degree of fit between worker needs and work setting, it can be expensive and time-consuming to implement. Perhaps for that reason, the companies we studied used this approach in only certain parts of the organization.

Providing Individual Choice

Most decisions about the design of group work spaces are made by the organization. In contrast, some companies offer their knowledge workers high degrees of choice about how to configure their own work environment within the constraints of a designated group work setting. Granting such options preserves a much-valued feeling of autonomy and generally increases knowledge-worker satisfaction, according to previous research and our anecdotal findings. Again, we’ve identified three levels of choice.

Low Choice

Some organizations allow their employees little or no choice; their work environments — regardless of the level of segmentation — are fixed and immutable. Employees cannot configure their workstations or work at home as they see fit. In a company that uses “hoteling,” they may not even be able to bring in their own desk items. Like low segmentation, low-choice environments are easy to manage, but they don’t necessarily save money — consider a low-choice environment consisting of plush private offices, for example.

Unexpected costs may also be incurred, as a ChiatDay office discovered in the early 1990s when it experimented with a low-choice virtual work environment. As employees tried to adjust to the new conditions, both productivity and retention plummeted.

Moderate Choice

Some companies will allow their employees to make some choices, but only within a menu of defined options. A large proportion of our research participants fit into this category. Sun Microsystems provides its knowledge workers with a menu of physical work environments (such as private offices, shared team rooms, satellite centers and work-from-home options), as well as hardware and software options. Workers are not given carte blanche — there are real budgetary limitations, especially in these times — but every effort is made to align work styles and preferences with work environments.

High Choice

A few companies allow their knowledge workers to determine numerous aspects of their own work settings. Employees may be encouraged to purchase their own furniture or technology solutions (special software, for example, needed by a particular team) and get reimbursed later. ChiatDay dropped the radical office design that gave employees little choice, and staff can now bring in all kinds of objects that might inspire or amuse them — from surfboards to their own pets.17 The freedom to customize one’s own work environment is perhaps most critical for highly creative workers such as those found at ChiatDay and IDEO.

Some companies use only one of the nine possible work-setting solutions in the framework and apply it throughout the organization. IDEO, for example, uses a “mass-personalized” solution in all its offices. It combines low segmentation (all employees share the same standard work setting) with high choice (employees are encouraged to bring in their own creative accoutrements). Nortel Networks applies the “configurable categories” solution throughout the company. Nortel defines a work setting group-by-group, but that setting can be further tailored in a modular way by individual employees.

Other companies employ more than one work-setting solution. Fidelity uses three. It has rolled out “mandatory specialization” solutions (high segmentation, low choice) and “modular made-to-order” solutions (high segmentation, moderate choice) to about half of its work force. Yet because of its legacy of universal planning — a workplace-design trend intended to reduce costs by introducing identical buildings filled with identical cubicles — the remainder of Fidelity’s employees still work in a “one-size-fits-all” work setting (low segmentation, low choice). The firm hopes to gradually convert all its work settings to ones more tailored to the type of work being performed.

Choosing a Solution

Which solution, or combination of solutions, is best for your organization? That depends on the answers to these questions:

How homogeneous is your organization?

Organizations whose knowledge workers share common work styles and needs, such as some architecture or law firms, may find that a one-size-fits-all solution works best. But for organizations whose workers’ needs vary, such a solution may significantly reduce employees’ effectiveness.

How important is it for your organization to align knowledge workers’ needs and their work settings?

The answer to this question depends on what drives the success of the organization. Managers who are more concerned with reinforcing a common value system or work process will prefer solutions in the lower left quadrant of the matrix. Those who need to foster the talents of workers engaged in tasks that are by nature difficult to program — such as are found in R&D units — should look for solutions in the upper right quadrant.

What level of resources are you willing to dedicate?

High degrees of segmentation require greater resources but can also lead to greater payoffs in enhanced performance. Companies that want to get the biggest bang for their buck may choose to focus on mass-personalized solutions. A low degree of segmentation means the environment is less complex and costly, but a high level of choice gives knowledge workers the autonomy they desire. For high-end knowledge workers, a high level of segmentation and choice is, in our view, the optimal solution.

Whichever solution one chooses, it won’t work if it isn’t carefully implemented. Any solution must be based on a thorough understanding of work styles and processes. Managers at Nortel, in fact, spent a year researching work processes through interviews and observations before identifiying eight team types and work settings. It must be remembered, too, that all solutions eventually fail. Organizations must therefore monitor the changing needs of their knowledge workers and keep abreast of how new technologies may enable new ways of working.

No one has all the answers on how to improve knowledge work, but managers shouldn’t feel paralyzed. They are correct not to attempt to engineer or program knowledge work, but that doesn’t mean such work lacks structure, cyclicality or leverage points for change. The keys are to maintain a balance between process and practice, to treat workers doing different kinds of work in appropriate ways, and to focus on more than simply hiring better knowledge workers.

While people may already be tired of hearing about “the knowledge age” or “the knowledge economy,” the surface of these concepts has barely been scratched. And organizations can’t begin to increase their understanding of what makes knowledge workers effective until they recognize the importance of these workers as a whole and how to differentiate among them as individuals. Only then can they begin to apply and combine the tools and approaches now being developed in a manner that is both cost-effective for the company and conducive to high performance in their employees.