Has enterprise software become too complex to be effective?
Technology has always been about hope. Since the beginning of the industrial revolution, businesses have embraced new technologies enthusiastically, and their optimism has been rewarded with improved processes, lower costs and reduced workforces. As the pace of technological innovation has intensified over the past two decades, businesses have come to expect that the next new thing will inevitably bring them larger market opportunities and bigger profits. Software, a technology so invisible and obscure to most of us that it appears to work like magic, especially lends itself to this kind of open-ended hope.
Software promises evolutions, revolutions and even transformations in how companies do business. The triumphant vision many buy into is that enterprise software in large organizations is fully integrated and intelligently controls infinitely complex business processes while remaining flexible enough to adapt to changing business needs. This vision of software lies at the core of what Thomas Friedman in “The World Is Flat” calls “the Wal-Mart Symphony in multiple movements — with no finale. It just plays over and over 24/7/365.”1 Whole systems march in lock step, providing synchronized, fully coordinated supply chains, production lines and services, just like a world-class orchestra. From online web orders through fulfillment, delivery, billing and customer service — the entire enterprise, organized end to end — that has been the promise. The age of smart machines would seem to be upon us.
Or is it? While a few companies like Wal-Mart Stores Inc. have achieved something close to that ideal, the way most large organizations actually process information belies that glorious vision and reveals a looking-glass world, where everything is in fact the opposite of what one might expect. Back-office systems — including both software applications and the data they process — are a variegated patchwork of systems, containing 50 or more databases and hundreds of separate software programs installed over decades and interconnected by idiosyncratic, Byzantine and poorly documented customized processes. To manage this growing complexity, IT departments have grown substantially: As a percentage of total investment, IT rose from 2.6% to 3.5% between 1970 and 1980.2 By 1990 IT consumed 9%, and by 1999 a whopping 22% of total investment went to IT. Growth in IT spending has fallen off, but it is nonetheless surprising to hear that today’s IT departments spend 70% to 80% of their budgets just trying to keep existing systems running.