Magazine Summer 2010 Research Feature

Is Decision-Based Evidence Making Necessarily Bad?

Many managers think they’ve committed their organizations to evidence-based decision making — but have instead, without realizing it, committed to decision-based evidence making. Is that all bad? What can be done to fix it?

Peter M. Tingling and Michael J. Brydon Reading Time: 16 min 

Topics

Permissions and PDF

Decision making is the essence of management, which explains why so much attention continues to be focused on how to do it better. In recent years, much has been written about evidence-based — or fact-based — decision making. The core idea is that decisions supported by hard facts and sound analysis are likely to be better than decisions made on the basis of instinct, folklore or informal anecdotal evidence. One need look no further than the shelves of the local bookstore to see an unprecedented collection of well-written titles extolling the virtues of data and analysis, such as Competing on Analytics (Davenport and Harris), Moneyball (Lewis) and Super Crunchers (Ayres). These books, like decision-making courses in business schools and the prescriptions of management consultants, focus on how to improve decision outcomes through improved process and technique. Many organizations have heeded the call and have invested heavily in data processing infrastructure and analytic tools, based on the assumption that better evidence-based decisions will follow naturally from these investments. While this focus on evidence is a welcome change from “thin slicing” or purely instinctive or intuitive snap judgments, these prescriptions tend to downplay the more fundamental questions: What is the relationship between evidence and the decision process that an organization actually uses? Why is evidence collected in the first place?

The Leading Question

Managers want ‘fact-based’ decisions. Are they getting them?

Findings
  • Evidence is not as frequent an input to decisions as suggested by the business press.
  • Not all decisions use evidence in the same way. Evidence can be used to make, inform or support a decision.
  • Managers need to be aware that evidence is shaped by subordinates to meet perceived expectations of company leaders.

Our research and consulting experience suggests that evidence is not as frequent an input to a decision process as suggested by the popular press. For example, we recently studied a major North American financial institution as it considered a proposal to change its enterprise e-mail platform from one technology to another. The organization had conducted two prior reviews of e-mail systems from major vendors and had twice recommended remaining with the existing supplier. However, the head of a small but influential and profitable division of the company advocated switching platforms in order to provide better integration with a specialized tool used only within his division.

Topics

About the Authors

Peter M. Tingling is founder and CEO of Octothorpe Software Corp. Michael J. Brydon is an associate professor of management at Simon Fraser University.

Tags:

Reprint #:

51419

More Like This

Add a comment

You must to post a comment.

First time here? Sign up for a free account: Comment on articles and get access to many more articles.

Comments (10)
DANIEL BOBKE
I think the title should be "Is EVIDENCE-Based DECISION Making Necessarily Bad?" - not "Is Decision-Based Evidence Making Necessarily Bad?".  Not sure what "evidence making" is - sounds like something attorneys do.
PHIL FRIEDRICH
Absolutely matches my experience.
Gerry La Londe-Berg
Is the "streetlight" reference at the end a wink to Gary Klien?  (Streetlights and Shadows).  Your thoughts are useful and not inconsistent with Mr. Klien's work.
Loretta Mahon Smith
"Decision making is the essence of management" is a great lead-in, and the article ends with a 4-point prescription for improving the process.

But the 5th point is missing.

5.  Transparency of evidence should always be included.  The data that is the basis for any decisions needs to be disclosed, and the quality and scope of that data should be shared with every audience.

The original audience of a report, or chart, or illustration may not be the most important reader!  Academics understand that citations are a critical component to successful works of research that are written.  Business needs to leverage this approach; every fact-based decision should have appropriate support easily available.

But in many cases these reports are delivered through application systems.  In the world of technology, you can find many best practices around Business Intelligence delivery, that will help provide guidance for the technology equivalent to citations in reporting (drill-down).  The DAMA Guide to the Data Management Body of Knowledge includes shared experience from more than 100 data management professionals worldwide.
Charles H. Green
Excellent analysis of a pervasive and ill-understood issue: how decisions are made in business.

In my work with professional services and complex intangibles businesses, there is a strong inclination to believe that firms use evidence to make decisions (your case 1), both internally, and in the case of clients buying their services.  In truth, your case 3, using evidence to support a decision, is far more common. 

The use of evidence to support decisions is common enough in sales in general that author Jeffrey Gitomer has slimmed it down to, "People buy with their heart, and rationalize it with their brains."  Anyone remember, "No one ever got fired for hiring IBM?"

I don't disagree with your eminently sensible conclusion that "it depends" on the role of evidence as to what we should do about it.  However, I think in your zeal perhaps to be even-handed (or perhaps to instinctively make evidence more supportive of decisions), you under-estimate the power, frequency and validity of decision-based evidence making. 

I'd suggest that decision-based evidence-making is more frequent, more valid, more correct, more efficient, and all in all more effective than I think your article suggests.  We have systemically and massively come to believe that business both is, and ought to be, highly rational. It is not.  Human decision-making remains wildly mysterious, even while the idea persists that if we could only reduce it to better decision-trees, dispassionate analyses, and the like, things would be better. Meanwhile, more and more studies suggest the primacy of gut instinct over our more leisurely cognitive analytics. 

Leadership, buying decisions, managerial approaches: what we really value are not analytically skilled people, but those who operate from principles and who can quickly apply those principles to the issues at hand. 

Love your analysis; but the prescription you provide boils down to--a process of more analysis. I'm not sure we need it.
Mary Federico
Fascinating and illuminating article -- thank you!

I will say, however, that from the point of view of a consultant in organizational behavior, there is nothing at all surprising about "the resistance of many managers to rational and analytical decision-making techniques." 

Among the many reasons for this resistance are the mental mechanisms of the confirmation bias (the "believing is seeing" problem in which people neither seek out nor notice disconfirming information), and cognitive dissonance (whereby those confronted with evidence that disconfirms what they already "know" -- particulalry if they've stated it publicly -- tend to discount the new evidence rather than change their previous positions). 

For 10 years I have worked with a broad array of companies instituting large-scale process-improvement initiatives (e.g., Six Sigma, Lean) that are designed to increase the use of evidence-based decision making, and I've seen these mental mechanisms in play time and time again.
Satyabroto Banerji
Structured decision-making promotes transparency, fair-play, and HRD, all at the same time. The corporation for which I worked was revolutionized once we adopted the Kepner-Tregoe protocol.
Alana Cates
When conscious decisions only represent a tiny fraction of the decisions made, and human nature circumvents fact based decisions, making better decisions is a matter of attitude.
Great leaders keep cognitive dissonance, confirmation bias, and skewed perspectives in check with empathy, humility, disassociation, acceptance and curiosity.
Mahesh K Enjeti
One way of determining when to accept or reject disconfirming evidence could be to analyse instances where this has occurred in the past and see what learnings can be drawn from such experiences.  For instance, in the illustration where the chair became a success could it be that the research did not cover aspects that may have explained the reasons for its subsequent  popularity.  There seems to be a reluctance in organisations to re-visit both good and bad decisions once the urgency of the decision making is over.  There is little attempt to objectively, dispassionately and unhurriedly evaluate the choices made. Compiling more evidence on both evidence based decision making and decision based evidence making will turn the lamp post that serves as a prop into an illumination source that lights up the path of choice.
Walter P. Blass
Amen! Making up evidence for a decision that top management wants is indeed rife. My best example is the purchase in 1983 by AT&T of Philips' telecommunications division in Hilversum. My group in Strategic Planning was asked to look at the proposition, and ended up warning management that the deal had some serious risks, especially should the Dutch Guilder drop in value,or if ever we had to lay off people, given Dutch law requiring huge severance payments for long time employees. I was visited by a "messenger" from inside the Company and advised to go along since "it was well known that the Vice-Chairman favored the deal" and I would lose any chance for promotion if I did not go along. I balked and sure enough, years later we lost a bundle because the deal was consumated but our fears came true.