Why Don’t We Know More About Knowledge?

More than 15 years ago, Peter Drucker heralded the beginning of the knowledge era. Since then, companies have made many attempts to leverage what they know and to increase their workers’ productivity. In order to bring together vast amounts of explicit knowledge, they have invested large sums in databases and content repositories; in order to help people track down others with tacit expertise, they have experimented with open offices, mobile technologies and online directories.

Some of this has helped; much of it has been a waste of resources. In fact, five years ago Drucker likened our understanding of knowledge-worker productivity today to our understanding of manual-labor productivity in 1900. Translation: We’ve got a long way to go.

To reorient managers more fruitfully, SMR asked three leading management thinkers to explain what we’ve learned and how we can do better in the future. Their contrarian responses bring clarity from a bird’s-eye view, while also suggesting ways to begin making progress on the ground.

Focus on the Process

In recent years, companies have tried a variety of approaches to resolving the supposed intractability of improved knowledge-worker productivity — from hiring chief knowledge officers to establishing knowledge-management programs. None of these efforts helped very much and most have now faded into obscurity.

But we should not despair. In fact, we actually know quite a lot about knowledge workers and their productivity. We know, for example, that the old dichotomy between manual workers and knowledge workers is not very meaningful. The best definition I have found for “knowledge worker” comes (unsurprisingly) from Drucker himself: “someone who knows more about his or her job than anyone else in the organization.” By this definition, the manufacturing worker who diagnoses and solves production problems, the utility linesman who schedules his own day and the warehouse worker who evaluates vendor performance all perform knowledge work and must be considered at least in part to be knowledge workers. Such people are increasingly the norm rather than the exception; fewer and fewer workers perform routine tasks that do not draw upon accumulated knowledge and expertise. To paraphrase Richard Nixon, “We are all knowledge workers now.”

A second important discovery is that the question “How do we increase knowledge-worker productivity?” is the wrong one to ask. The very concept of knowledge-worker productivity has little meaning. What output does a knowledge worker produce that could be used to measure his or her productivity? Decisions? Reports? Analyses? And would more of them be an improvement?

The focus on knowledge workers as isolated individuals whose personal productivity must be improved is a Taylorite relic. The goal is not to get more out of individuals but to get more out of the entire organization, and the way to do that is by improving the performance of the end-to-end business processes to which individual workers of every stripe contribute. Knowledge workers are part of many different business processes, from product development to customer service, and these processes have unique metrics and objectives. There is no universal approach or standard tool to improve the productivity of a generically defined knowledge worker. Instead, each business process needs to be examined individually and redesigned on its own terms.

A frontal assault on individual tasks is not the way to improve knowledge-work processes. The key, rather, is to eliminate the enormous amount of non-value-adding work that gets in the way. People spend inordinate amounts of time on activities that do not directly contribute to a process’s outcome and are therefore of no value to the customer: waiting for the arrival of inputs, getting clarification on ambiguous goals or information, redoing work in response to changes in constraints or context, arguing about priorities and so on. These activities are inevitable artifacts of the ways in which work has long been organized and performed.

Moreover, they cannot simply be eliminated by fiat. Remove the non-value-added work from an existing process, and it will grind to a halt. Nor is technology of much help. Such work needs to be designed out. The productive activities of the process need to be reassigned, resequenced and relocated so that there is less occasion for non-value-adding work to arise in the first place.

For instance, product development can be transformed in the following way: Assign each product to a cross-functional team with representatives from relevant departments; hold team members jointly accountable for results; conduct formal project assessments at regular intervals; ensure that a project does not proceed until it has reached closure on a specified set of issues; and terminate projects that do not continue to meet criteria for eventual success. When such a process is in place, no time and energy are wasted in miscommunications between departments; there is no finger-pointing or pursuit of incompatible goals; and effort is not expended on dead-end projects. As a result, organizations adopting this process commonly see that development time falls by 75%, aggregate development expenditures shrink by 50%, and customer satisfaction increases markedly. Considering that we’re not supposed to know much about increasing knowledge-worker productivity, that’s not so bad.

Some knowledge workers bristle at the notion of process — they see it as an intrusion on their creativity and individuality. This response betrays a misunderstanding of the nature of process. Process is not about the routinization and bureaucratization of work but about positioning all individual activities in the larger context in which they are performed. A process specifies which steps must be performed, by whom, where, in what order and so on; it does not specify how each step is to be performed. A sales process, for example, does not specify how a sales rep must interact with a customer. It does specify that the rep must qualify the customer early on, assess the profitability of the opportunity, and understand the costs of meeting the customer’s needs before quoting a price. Process leverages the ability and creativity of knowledge workers and ensures that their time is used well.

Given all that we have learned in the past decade or so about how work gets done, it’s time to stop berating ourselves about our lack of progress in increasing knowledge-worker productivity. In fact, it may be time to retire the term “knowledge worker” itself.

Seek Out “Deep Smarts”

In the past several years, knowledge-management theorists have gone back to the caves. That is, they have become less enamored of technology and have rediscovered the value of our humanity. Although our archives are computers rather than cave walls, modern people are like the cave dwellers in certain important respects. We still love stories and learning in communities; we still want to establish trust before conveying information; we still crave experience-based, proven and practical knowledge.

But attempts to meet such basic needs require behavior that is out of sync with the pace of modern life. We want context, but we have to deliver bullet points. We want the exquisitely powerful guidance of another mind through a maze of data, but we want it immediately and we want it focused on our particular needs. We want to learn from each other, but not if it takes more than a few hours and not if it requires struggling for answers. In short, our brains create and absorb knowledge the way they always have, but we seek ways around their limitations. Technology has immensely improved access to, and transmission of, information, but it cannot create shortcuts to the most valuable kinds of knowledge. That dilemma explains much about why organizations still have trouble managing knowledge.

What can managers do about this problem? They can start by rectifying a common mistake: overlooking what I call “deep smarts” in people and protecting “shallow smarts.” In giving short shrift to the former, companies frequently sacrifice some important experience-based knowledge; in overvaluing the latter, they treat some skills that are readily available on the open market as unique organizational assets.

People with deep smarts draw on a huge store of tacit knowledge, built through years of experience. They have many of the characteristics of any expert: the ability to make swift decisions on the basis of pattern recognition, to extrapolate from the known to the possible and to make subtle distinctions that are invisible to a novice. They are also able to take a systems view of a complex product, an organization or an environment — and to predict interactions and interdependencies. In other words, possessors of deep smarts have the mental capability of a satellite: They can fly over the landscape, grasp the overall situation and then zoom in on critical details and potential problems.

As organizations confront a significant shortfall in both technical and management skills — a challenge that will intensify as the Baby Boomers begin to retire — how will they capture and transfer the essential wisdom that people with deep smarts have acquired over the years? Technology will help, of course. But this time, before companies spend millions on computerized archives that will rarely be used, managers should bear in mind the difficulty of transferring tacit knowledge. It is not necessary to go all the way back to the caves for a model, but only to the Middle Ages, a time when apprentices learned from master craftsmen. Then as now, tacit knowledge was not transferred from one brain to another; it was re-created through experience. Today, knowledge coaches — experts who are motivated and capable of helping relative novices learn through guided experience — can speed up that recreation. Such coaches are not merely process facilitators; they have their own hard-won know-how to pass along. Knowledge coaches operate not by lecturing and not even by telling stories alone (although such methods help). Rather, they help their protégés build experience through guided practice, guided observation, guided problem solving and guided experimentation.

Managers who hope to leverage the tacit knowledge within their organizations need to create programs of guided experience and to train knowledge coaches. And such initiatives have to be treated as investments rather than merely as expenses. Unfortunately, because managers have been more thoroughly trained in finance than in learning processes, it is far easier to obtain funding for purchases of hardware and software than it is to obtain resources for investments in wetware (people). Companies tend to sacrifice efficacy in the name of efficiency. Those simple facts explain why organizations have not made more progress toward leveraging knowledge. Only when they begin to weld modern technologies to age-old facts of human behavior will they resolve this conundrum.

Learn From Experiments

We still have a vast amount to learn about knowledge work, but there is an established (or emerging) consensus about what we do know. First, the definition of “knowledge worker.” Although every job requires some knowledge, most would agree that knowledge workers are people with high levels of education and expertise whose primary task is the creation, distribution or application of knowledge. Second, not all knowledge workers are alike: They need to be segmented — by the degree of collaboration required to do the work or the level of expertise necessary to perform it, for example —before managers intervene in their work processes.

A third point of agreement: Knowledge workers like autonomy and don’t like to be told how to do their work. There are data to support this assertion, but it’s pretty obvious: Why go to the trouble of accumulating education and expertise only to have someone else tell you how to do your job? Corollaries of this point are that knowledge workers do not take kindly to top-down re-engineering of their jobs and that separating the design of knowledge work from its execution is a bad idea. That leads to a fourth point of consensus: Knowledge work is a matter of process — the design for how work is to be done — and practice, the way individual workers respond to the real world of work and accomplish their assigned tasks. To really understand work practice requires detailed observation and accepting that there are usually good reasons why workers do their jobs in a particular way.

In addition to these general points, researchers have taught us about particular types of knowledge workers. From Tom Allen’s research, for example, we’ve learned that scientists and engineers need to work very near each other in order to exchange ideas. From Paul Adler’s work, we know that it’s possible to combine process and practice in a way that makes software engineers happy. From Marshall Van Alstyne’s work, we know that executive searchers are more productive when they make effective use of electronic communications. And from my own research, we know that expert physicians can be persuaded to make effective use of knowledge management that’s embedded in order entry — doctors may not have time to search knowledge repositories, but they pay attention when the computer tells them, for example, that there’s a better drug or test available for a particular patient. I could go on and on with these narrow but important findings about specific types of knowledge workers.

These truths about knowledge work also begin to suggest what we don’t know — which is considerable. We rarely know how to combine process and practice in the right proportions for specific jobs and circumstances. For example, we know that physicians will need to follow different processes in the future, but it’s unclear how to get them to abandon existing practices. Nor do we know to what extent autonomy must be preserved in what kinds of professions and roles. And while we’ve learned about particular knowledge-work jobs and roles, we know little about whether these findings can be generalized. We don’t even have broadly applicable measures of knowledge-work productivity or quality.

The good news is that without too much more effort, companies could make substantial progress in resolving these issues. Every day, organizations conduct experiments on knowledge workers. They move knowledge workers from closed offices to cubicles or vice versa. They implement particular technologies and try new organizational approaches. Unfortunately, organizations learn little from these experiments. They undertake them not because they want to gain knowledge about how to make their workers more productive, but because they want to cut costs or because some vendor or fadmonger convinces them that the magic bullet for improving knowledge-worker performance is now available.

What would organizations do if they were serious about learning from knowledge-work experiments? They would measure the productivity, performance or satisfaction of knowledge workers before and after any changes. They would change only one thing at a time, making it possible to know which factor drove a change in outcomes. And they wouldn’t change anything at all for a control group.

The tenets of experimental design are well understood, if not always easy to apply. But lack of experimental discipline is causing us to miss out on the opportunity to learn a great deal. If we could harness all those sources of potential learning, our understanding of what makes knowledge workers tick would accelerate dramatically.