Magazine Spring 2014 Research Feature

The Pitfalls of Project Status Reporting

Will your next big IT project be on time and deliver what was promised? Maybe — but maybe not. Accepting five inconvenient truths about project status reporting can greatly reduce the chance of being blindsided by unpleasant surprises.

Mark Keil, H. Jeff Smith, Charalambos L. Iacovou and Ronald L. Thompson Reading Time: 17 min 

Topics

Permissions and PDF

Nobody anticipated the size and scope of the problems we experienced once the site launched.”

— White House spokesman Eric Schultz, commenting on the HealthCare.gov website.1

Missing the Target

All too often, executives are surprised when complex IT projects fail to meet goals.

The launch problems associated with the HealthCare.gov website — intended to provide a one-stop health insurance shopping portal for uninsured and underinsured Americans beginning October 1, 2013 — are now well-known. As Wall Street Journal reporter Clint Boulton wrote, “For the first couple weeks of October, it appeared that the Affordable Care Act, the centerpiece of the Obama Administration’s domestic policy, was in danger of being undone by an IT project gone wrong. HealthCare.gov, the $630 million project meant to showcase the administration’s tech savviness, was an end user nightmare.”2

What went wrong with the HealthCare.gov launch? It has been reported that the implementation process suffered from a lack of clear direction, repeated changes in requirements (some less than two weeks before the anticipated “go” date) and a severely cramped test schedule, which allowed little time to uncover and address integration issues.3 Even so, some lawmakers have reported that Congress had been told as late as September 2013 that the implementation project was on track. These lawmakers have “accused both the contractors and the [Obama] administration of withholding information about the looming failure.”4 It has been reported that an outside consulting firm “in late March [provided] a clear warning” that “foreshadowed many of the problems that have dogged HealthCare.gov since its rollout” and that some high-ranking White House officials were briefed regarding the report’s contents in April 2013.5

While the problems accompanying the HealthCare.gov site launch were unusually high-profile, they are not uncommon; all too frequently, leaders in the private sector as well as the public sector are caught by surprise when projects — particularly complex IT projects — run into trouble. But complex IT projects do not fail overnight; they fail one day at a time, and generally only after numerous warning signs. Our research suggests that for many executives, accepting five inconvenient truths about project reporting can greatly reduce the chance of being blindsided by a troubled project launch. (See “About the Research.

Topics

About the Authors

Mark Keil is the John B. Zellars Professor of Computer Information Systems at the J. Mack Robinson College of Business at Georgia State University in Atlanta. H. Jeff Smith is the George & Mildred Panuska Professor in Business at Miami University’s Farmer School of Business in Oxford, Ohio. Charalambos L. Iacovou is the vice dean and a professor of management at the School of Business at Wake Forest University in Winston-Salem, North Carolina. Ronald L. Thompson is a professor of management at the School of Business at Wake Forest University.

References

1. J. Eilperin and S. Somashekhar, “Private Consultants Warned of Risks Before HealthCare.Gov’s Oct. 1 Launch,” Washington Post, Nov. 18, 2013.

2. C. Boulton, “HealthCare.Gov’s Sickly Launch Defined Bad IT Projects in 2013,” December 24, 2013, http://blogs.wsj.com.

3. For example, see “Lessons in Project Management from the Obamacare Website,” Oct. 24, 2013, www.cosmoscon.com; and D.R. Hammons, “HealthCare.gov — Obamacare Website Project: A Retrospective,” December 2013, www.pmworldjournal.net.

4. J. Haberkorn and J. Millman, “Contractors Grilled on the Hill,” October 24, 2013, www.politico.com.

5. Eilperin and Somashekhar, “Private Consultants Warned of Risks.”

6. A.P. Snow, M. Keil and L. Wallace, “The Effects of Optimistic and Pessimistic Biasing on Software Project Status Reporting,” Information & Management 44, no. 2 (March 2007): 130-141.

7. B.C.Y. Tan, H.J. Smith, M. Keil and R. Montealegre, “Reporting Bad News About Software Projects: Impact of Organizational Climate and Information Asymmetry in an Individualistic and a Collectivist Culture,” IEEE Transactions on Engineering Management 50, no. 1 (February 2003): 64-77; M. Keil and D. Robey, “Blowing the Whistle on Troubled Software Projects,” Communications of the ACM 44, no. 4 (April 2001): 87-93; and H.J. Smith, M. Keil and G. Depledge, “Keeping Mum as the Project Goes Under: Towards and Explanatory Model,” Journal of Management Information Systems 18, no. 2 (fall 2001): 189-228.

8. C.W. Park, G. Im and M. Keil, “Overcoming the Mum Effect in IT Project Reporting: Impacts of Fault Responsibility and Time Urgency,” Journal of the Association for Information Systems 9, no. 7 (July 2008): 409-431.

9. Tan et al., “Reporting Bad News About Software Projects.”

10. H.J. Smith, R. Thompson and C. Iacovou, “The Impact of Ethical Climate on Project Status Misreporting,” Journal of Business Ethics 90, no. 4 (December 2009): 577-591; see also H.J. Smith, R.L. Thompson and C.L. Iacovou, “An Extended Model of Selective Status Reporting in Information Systems Projects,” unpublished ms, 2013.

11. Snow, Keil and Wallace, “Effects of Optimistic and Pessimistic Biasing.”

12. Smith, Thompson and Iacovou,“Impact of Ethical Climate.”

13. C.W. Park, M. Keil and J.W. Kim, “The Effect of IT Failure Impact and Personal Morality on IT Project Reporting Behavior,” IEEE Transactions on Engineering Management 56, no. 1 (February 2009): 45-60.

14. M. Keil, G.P. Im and M. Mähring, “Reporting Bad News on Software Projects: The Effects of Culturally Constituted Views of Face-Saving,” Information Systems Journal 17, no. 1 (January 2007): 59-87.

15. J.B. Cullen, B. Victor and J.W. Bronson, “The Ethical Climate Questionnaire: An Assessment of Its Development and Validity,” Psychological Reports 73, no. 2 (October 1993): 667-674.

16. M. Keil, H.J. Smith, C.L. Iacovou and R.L. Thompson, “The Dynamics of IT Project Status Reporting Process: A Self-Reinforcing Cycle of Mistrust,” unpublished ms, 2013.

17. M. Keil, A. Tiwana, R. Sainsbury and S. Sneha, “Toward a Theory of Whistleblowing Intentions: A Benefit-to-Cost Differential Perspective,” Decision Sciences 41, no. 4 (November 2010): 787-812.

18. C.L. Iacovou, R.L. Thompson and H.J. Smith, “Selective Status Reporting in Information Systems Projects: A Dyadic-Level Investigation,” MIS Quarterly 33, no. 4 (December 2009): 785-810.

19. Ibid.

20. Keil et al., “Toward a Theory of Whistleblowing Intentions.”

21. Keil and Robey, “Blowing the Whistle.”

22. C.L. Iacovou, “Managing IS Project Failures: A Project Management Perspective” (Ph.D. diss., University of British Columbia, 1999); C.W. Park and M. Keil, “Organizational Silence and Whistle-Blowing on IT Projects: An Integrated Model,” Decision Sciences 40, no. 4 (November 2009): 901-918; Keil and Robey, “Blowing the Whistle”; and M.J. Cuellar, M. Keil and R.D. Johnson, “The Deaf Effect Response to Bad News Reporting in Information Systems Projects,” e-Service Journal 5, no. 1 (September 2006): 75-97.

23. Iacovou, “Managing IS Project Failures.”

24. Cuellar, Keil and Johnson,“Deaf Effect Response.”

25. Keil and Robey, “Blowing the Whistle.”

i. These 14 studies are described in Snow, Keil and Wallace,“The Effects of Optimistic and Pessimistic Biasing”; Smith, Thompson and Iacovou, “Impact of Ethical Climate”; Smith, Thompson and Iacovou, “Extended Model of Selective Status Reporting;” Tan et al., “Reporting Bad News About Software Projects”; Keil et al., “Dynamics of IT Project Status Reporting”; Keil et al., “Toward a Theory of Whistleblowing Intentions”; Iacovou, Thompson and Smith, “Selective Status Reporting”; Iacovou, “Managing IS Project Failures”; Keil and Robey, “Blowing the Whistle”; Smith, Keil and Depledge, “Keeping Mum”; Keil, Im and Mähring, “Reporting Bad News on Software Projects”; Park, Im and Keil, “Overcoming the Mum Effect”; Park, Keil and Kim, “Effect of IT Failure Impact”; Park and Keil, “Organizational Silence and Whistle-Blowing”; and Cuellar, Keil and Johnson, “Deaf Effect Response.”

ii. Keil et al., “Dynamics of IT Project Status Reporting.”

iii. Ibid. This is not an exhaustive list.

Tags:

Reprint #:

55317

More Like This

Add a comment

You must to post a comment.

First time here? Sign up for a free account: Comment on articles and get access to many more articles.

Comments (6)
shubhra gupta
Thanks Mark for wonderful article. Illustrating the research finding with the help of a real world example (HealthCare.gov website case study) was very helpful for readers to understand the entire concept. Besides the points you have mentioned in the article, there can be many unforeseen circumstances which lead to delay. For example, if a project manager wants to quit/switch his job, to convey this to the client can be tricky. Most of the companies find replacement of current Project Manager and then convey it to the client. I am not going to judge whether this is right or wrong to do but I see some justifiable reasons to do this. The clients after investing so much money get way too much worried about wellbeing of project. Sometimes their reactions make lives of consulting companies even more difficult. In order to avoid all the confusions, the companies reveal the facts little late.
John Colangelo
This is a very informative article.  We have to train project team members in estimating procedures, then the team should conduct formal estimates following those procedures and publish their results.  PMs should reinforce the notion that a "Red" project status is not a punitive designation.  It means that one side of the project triangle is out of balance (or more) and the project requires immediate attention.  I also think it is key that the PSMO not be subordinate to any reporting structure underneath the executive body so that they can remain objective and not fear bias reporting.  Sometimes the burden to "call the baby ugly" falls upon the shoulders of the PM and he/she needs to feel safe doing so.  When I cut my teeth on project management years ago I learned the phrase "pay me now.........or pay me later" which basically is the same as "Good, Cheap, Fast"  Pick Two!    Complex projects require detailed analysis and comprehensive engagement and awareness.  It starts with solid estimating and continues with excellent PM oversight.
ian
Elmar,  My reading of what Gary said was not so much generating a number, but the expectation that a specific estimate will be "right".  This makes the task impossible, not just difficult!  

Estimates have a valuable contribution in planning - "Planning is priceless, plans are worthless".  The problem comes when someone turns estimates into deadlines and milestones appear.  Senior managers believe that focusing on hitting each milestone is the best way to hit the project target, and so estimates become commitments.  Early finishes get hidden, team member trust reduces...downward spiral.

On that basis I think Gary was right - (i) the authors address symptoms more than root causes, and (ii) trying to predict the future and treat estimates as commitments, is very damaging to projects .  

That doesn't mean there is no role for asking "are we on course", and managing a project to meet the project's commitments.
ian
Well said Gary.

The implication in the recommendations is that the problem lies in the people.  Maybe there are more fundamental issues, and some root causes that should be considered, including the underlying process of planning and managing projects.  

I suggest Inconvenient Truth #6 - You cant predict the future, and even with simple repeat activities task time and cost follows a statistical distribution.  

The best place to manage this uncertainty is a the aggregate (project) level - in the same way the insurance industry works.  If you try and manage it at the task or work package level by demanding team members (and suppliers) give you fixed and certain delivery dates, your projects will take longer and be much less likely to be on time or on budget.  

Resolution:  Plan tasks based on 50% probable time/cost, add strategic project time and cost buffers to protect against the uncertainty.  Focus team members on completing as quickly as possible.  Use the amount of buffer consumption relative to overall progress as the single project control measure.  The project management approach known as Critical Chain Project Management (CCPM) embeds these principles.

CCPM has been used not just in IT, but most other sectors, with great success all over the world.  Major ERP system implementations have used it to bring late projects back on track, and reduce costs.  Leading organisations have developed hybrid Agile & CCPM techniques to take advantage of both.

Although a method such as CCPM in itself can't guarantee success, without it you sure have your work cut out even to hit your targets, let alone get faster and cheaper without cutting corners.
Elmar Roberg
I don't agree. Estimating may be hard work, but it is not difficult. An estimate is, by definition an estimate, not reality.
Tom de Marco hit the nail on the head in an article in 1993 IEEE Software Magazine, March 1993 edition, entitled, "Why does software cost so much?"
Do yourself a favour, get hold of the collection of his essays in a book "Why does software cost so much? and other puzzles of the information age." Dorset House, 1995.
After 21 years, it is really sad to see how little progress we have made. Sigh.
In the book, he has another little article "mad about metrics" (read that title carefully, better still, get the article and read it.)
GARY LLOYD
Although I have no quibble with what the authors says about the difficulty of establishing open and honest reporting for large IT projects, I believe they are addressing symptoms rather than causes. 

The fundamental problem is that is simply impossible to forecast costs and schedule with any accuracy. Suppliers and clients are, however, locked into a a mutual deception that it is and put in place processes to create and monitor plans.

What is needed is an adult conversation about the difficulty of estimating, leading to a value-based approach to design and delivery, in which usable chunks of business value are delivered on a regular basis. Rather than asking, "are we on course", we are reviewing the value, cost, quality, value and working relationship are regular intervals.