Will Web Services Really Transform Collaboration?

For corporate computing, the big story of the new millennium has been Web services. As the excitement around the Y2K “crisis” and e-commerce business models faded just after the turn of the century, enthusiasm started to build for this technology, which is a set of tools that make it easier for applications to talk to each other. In other words, Web services install the plumbing required for information systems to interact without human involvement. An important aspect of Web services is that they work as well between companies as within them. Just as the Internet and the World Wide Web led to a huge change in how people interact with distant applications (for example, the applications at Amazon.com), so the Internet and Web services hold out the promise of drastically changing how distant applications interact with each other. See sidebar

But how real is this promise? Will Web services be a truly disruptive innovation that revolutionizes how companies interact? Or will they turn out to be the latest in a series of technologies, developed and promoted by the highly entrepreneurial IT industries, that fizzle when they hit the real world? (You may remember CASE tools and Internet “push” technology.1)

The answer is that they will be neither. They will be highly useful, but not quickly or obviously disruptive; in fact, they will reinforce existing relationships rather than catalyze new ones. What’s more, the application-integration challenges that remain unaddressed by Web services are the really difficult ones and can only be overcome by the work of managers and leaders, not technologists or consortia.

The Promise of Web Services

At least five different communities — software vendors, computer scientists, technology analysts, business pundits and industry consortia — are currently generating enthusiasm about Web services, which are defined here as open, Internet-era standards for exchanging data between applications.

“With the Web services architecture, tight couplings will be replaced with loose couplings,” assert management consultant John Hagel and computer scientist John Seely Brown. “Because everyone will share the same standards for data description and connection protocols, applications will be able to talk freely with other applications, without costly reprogramming. This will make it much easier for companies to shift operations and partnerships in response to market or competitive stimuli.” Hagel and Brown further conclude that “Over time, the location of particular capabilities — whether inside or outside the walls of any given company — will become less important than the ability to discover and orchestrate distinctive capabilities across enterprises in order to deliver greater value to customers. In the process, many companies will find themselves turned inside out, with their formerly well-guarded core capabilities visible and accessible to all.”2

These and other voices stress two points: that Web services allow construction of modular and interchangeable building blocks of software, and that their impact — on how companies collaborate and compete — will be profound. The first of these two trends is already well under way, but the second is not. Right now, there are a huge number of loose couplings between companies all over the world thanks to modern IT. But they’re not application-to-application links; there are always people involved. They use the technologies of the network era — Web servers and clients, e-mail and instant-message applications, groupware such as Lotus Notes/Domino, and nascent “social software” such as wikis, Friendster.com and Linkedin.com. The connections enabled by these technologies are dynamic, fluid, easy to make and break, and often highly productive and useful. In some of them, a person is present on only “one side of the conversation,” while an application is present on the other side. Surfing the Web is probably the most common human-to-application interaction taking place over the Internet. During other interactions, such as e-mail and IM, there are people on both sides of the conversation.

While we should not underestimate the importance of these technologies, which are clearly fostering collaboration and boosting productivity, we should also not overestimate their impact on the business world.

It’s true that the Web has given rise to important companies such as Amazon, eBay, Yahoo and Google. And it’s also true that a few sectors, such as travel, bookselling and music distribution, have been seriously shaken up by Internet technologies. It is also likely that the new tool kit for human-to-human and human-to-application interaction will lead to a surge in outsourcing of some kinds of knowledge-based work.

Despite these important changes, however, most companies continue to work with familiar partners in familiar ways. They probably encourage their employees to book travel online, and they might be thinking about outsourcing a call center to India, but they’re still dealing with the same populations of customers, suppliers, joint-venture partners, financiers, consultants, auditors and so on that existed five or 10 years ago. In other words, their value chains and industry structures have remained constant and havenot been turned inside out by the Web.3 What’s more, the companies widely recognized as effective orchestrators of far-flung supply chains — including Dell, Cisco, Li & Fung, Wal-Mart and Arrow Electronics — were all established well before the Internet era.

The Realities of Systems Integration

Advocates of Web services claim that their technology is more revolutionary because it enables application-to-application interactions. In one sense, there is ample justification for this claim. If information systems could look for and find each other, share data and execute business processes, all with no (or very little) human involvement, we would probably find ourselves in a very different business world.

Web services, however, will not create this world, nor will any technology on the horizon.

The points of agreement that underlie any business interaction using computers and networks can be grouped into three successive levels. (See “Levels of Agreement for Computer-Mediated Interaction.”) At the lowest level, Level 1, are agreements about how totransport data from one endpoint of the interaction to the other(s). Decisions made at this level include the network to be used, the encryption (if any) employed, and how to “wrap” the transmission so it will be readable by the destination computer. At Level 2 are agreements about thepayload, or the contents of each transmission. Decisions at this level include the family of e-documents to be exchanged (will we exchange purchase orders, invoices or both?) and contents of each e-document (what information is included in a purchase order; where does each field go; what currencies are acceptable; and how do we indicate which currency is being used?). At Level 3, if the e-document is being used as part of a multistep businessprocess, collaborators must agree on its parameters (what are the steps and who executes each one; what are the possible branches and endpoints; and what happens if there’s an error or an exception?).

The Advantages of Having People at the Interfaces

It is important to note that only the lowest level of agreement, the transport level, must be in place before human-human or human-application interactions can take place. As soon as the Web’s engineers finalized its transport protocols, the rest of us quickly began using it to send messages to each other and browse Web sites. We didn’t need to sit down in advance and agree on whether sentences in e-mails would always begin with capital letters or on whether all e-commerce sites would include shopping carts and personalized recommendations.

This is because humans are extremely flexible, forgiving and intuitive, at least compared with computers. We don’t need to know in advance precisely what e-documents we’ll be receiving or what processes we’ll be executing, and we don’t get bothered when an e-document or a process doesn’t correspond exactly to what we’re expecting. (There are limits, of course. If I get a phone bill for $1 million instead of $100, I’m certainly not going to continue the payment process.) Once a good transport network is in place — and thanks to the Internet and the Web, we have one —people can start interacting with applications and with each other, using the tools listed above and reaping the benefits from them. And this is exactly what we are doing.

Lots of Rules and No Mercy

For application-application interactions, however, the situation is radically different. This is because, as the mythologist Joseph Campbell has observed, “Computers are like Old Testament gods; lots of rules and no mercy.” If two applications want to send and receive information, they must first share a complete and exact agreement about all the e-documents they will exchange; they must, in other words, have complete Level 1and Level 2 agreements upfront. Otherwise, the transmission will simply fail. And if the applications want to do more than simply send discrete transmissions back and forth — if they want to execute entire business processes — they must first finalize all Level 3 agreements as well.

The problem is that no two applications will start with exactly the same Level 2 or Level 3 configurations. The applications used in different departments or different locations of a single company, for example, are virtually guaranteed to have inconsistency in customer lists, part-numbering schemes, approved suppliers, addresses for these suppliers and so on. Computer scientists term this the “corporate household” problem.4 It is a major headache during any internal systems-integration effort, and the problem is certainly worse between companies than within them.

The situation is even more fragmented at the level of the business process. Even though many companies have purchased enterprise information systems with preconfigured business-process templates, most have modified the templates during implementation. As a result, the business processes they execute are so dissimilar that no two companies could simply hook together their systems and expect them to carry out Level 3 interactions.

Web services by themselves don’t address the household problem at all. The technologies that make up the current Web-services tool kit work at Level 1; they make it possible for two applications to talk to each other, but theydon’t specify what conversations they should have (Level 3) or what words they should use (Level 2). XML, for example, is a core element of Web services; it’s the language used to build e-documents for application-application interactions, just like HTML is used to build Web pages for human-application interactions. But just as HTML doesn’t predetermine what kind of pictures or text should go on a page (Level 2) or in what order pages should be viewed (Level 3), XML doesn’t predefine the set of valid e-documents (Level 2) or business processes (Level 3). HTML enables human-application interactions, but doesn’t impose them. Web services playexactly the same role for application-application interactions.

What Can Be Done?

So can anything be done about the household problem? Two types of large-scale solution are being proposed. The first comes from the computer-science community and essentially comprises middle-ware that converts companies’ existing Level 2 and Level 3 configurations as required. While many of these proposed solutions are elegant, they suffer from two serious shortcomings. First, they work only if all participants agree to use the same solution, which seems unlikely since it was theirlack of agreement that necessitated a technical solution in the first place. Second, no automated conversion rule will be right all the time. Businesses are not going to hand over a critical process to interacting applications if there’s a real chance that the applications will get it wrong. As a result, purely technical solutions to the household problem are not going to successfully move from the lab to the real world of corporate computing, even if we assume away the critical issues of security and fraud.

The second category of solution comes from industry-level consortia. These are nonprofit organizations, founded and funded by companies in an industry. Their goal is to propose Level 2 and Level 3 standards for application-application interaction that will be both useful and acceptable to participants. One of the oldest and best-known consortia is the high-tech industry’s RosettaNet. Consortia can do valuable work but can’t come close to eliminating the household problem by themselves. They move slowly (because of their wide membership and the need to make decisions by consensus), concentrate on the least controversial and proprietary standards, and have no enforcement power. They are the starting point, rather than the finishing line, for B2B projects, which are defined here as efforts to build application-application links between companies and which, today, usually employ Web-services technologies.

Case Study: IBM Takes a B2B Journey

So how do efforts to build such application-application interactions actually advance? And will there come a “tipping point” after which they become much easier to implement and spread quickly? I recently studied a B2B effort that suggests answers to these questions.5

At IBM Corp., the EMEA organization (which covers Europe, the Middle East and Africa) has been working with independent distributors in several countries to automate the ordering of midrange computing systems. The midrange B2B ordering effort was born out of a desire to standardize and improve business processes between IBM and its distributors. In 2000, executives at Magirus, a distributor of information-technology systems based in Stuttgart, Germany, were being increasingly vocal with complaints that IBM was difficult to do business with and that its processes were inconsistent across different product families and geographies. At that time, distributors were using a Web site to place midrange orders (in other words, human-application interactions were already taking place), but Magirus and members of IBM’s Business Partner Service Organization (BPSO) wanted to try something more digitally ambitious.

They began an effort to place orders using electronic data interchange (EDI), a pre-Internet B2B technology, but abandoned it after more than a year of effort. Midrange systems were simply too complex, and IBM’s internal IT infrastructure was in too much flux for EDI to work. The EDI project leader from the IBM side, Didier Boullery, then began to learn more about XML and about RosettaNet, which had defined a Level 3 Partner Interaction Process (PIP) for ordering computer systems. During the same time, IBM made important internal IT decisions and stabilized its infrastructure, and Boullery felt that there was an opportunity to try midrange B2B ordering again and to get it right. He convinced Magirus to keep working toward B2B.

Convincing IBM was, in many ways, harder. EMEA was a large and complex organization, with some groups organized by product, some by geography, some by customer segment and some by functional area. This meant that internal entrepreneurs like Boullery had many possible sources of sponsorship, but also that they had to persuade all groups from which they wanted people, funding, permissions or commitments. Boullery found Jamie Hewitt, an executive within the IBM group responsible for making and selling midrange systems, to be especially helpful; she had seen successful B2B efforts within IBM’s U.S. operations and was eager to expand this work to Europe. She provided funds and also helped convince her European colleagues to make people and money available. Boullery was eventually able to put together a cross-functional team for the Magirus project.

IBM was a member of RosettaNet and had officially stated its intentions to use the PIPs developed by the consortium in its B2B efforts. The team found, however, that RosettaNet’s standard ordering PIP was unsuitable. Midrange systems were too complex, and IBM’s product nomenclature too idiosyncratic, for the rather basic PIP. The team therefore modified it significantly to incorporate the additional information and steps required.

After finding that most commercial B2B software was overengineered for its needs, Magirus bought inexpensive software that enabled XML transmissions. The company also worked with an external XML gateway provider, which ensured the delivery and integrity of each transmission; IBM worked with a different external gateway provider, so each e-document actually passed through four companies: IBM, Magirus, and the gateway providers for each company.

In May 2003 the new technology went live, and since then all of Magirus’s midrange orders have gone directly from its internal applications to IBM’s. There is some quantitative evidence that these orders are cleaner than others and that they get processed more quickly. But neither IBM nor Magirus currently views the midrange B2B ordering project as a standalone success, either in financial or operational terms. Instead, they consider it a first step toward broader and deeper application-application interactions. Recently, for example, Magirus has started placing orders for IBM software using the B2B channel. In late 2003, four other European distributors of midrange systems agreed to work with IBM toward B2B ordering. By November of 2004, three were already live with the new technology.

Lessons of the IBM Case

The IBM case study is a particularly interesting one because it shows an effort to go beyond the capabilities of previous B2B technologies like EDI, and to define Level 2 and Level 3 agreements where none existed previously. Four lessons from this effort stand out.

First, the work of creating new application-application interactions is relatively slow and requires sustained attention and commitment from all participants; it took more than a year for Magirus and IBM to automate one business process. Second, there is no silver-bullet technology that can greatly accelerate this work; it took time to persuade all of the people and groups that needed to be involved and to defineexactly how applications were going to interact at Level 2 and Level 3 during midrange ordering. The tool kit of computer science, large though it is, cannot help much with either of these. Third, the RosettaNet consortium’s ordering PIP did help, but the team needed to greatly modify it and felt free to do so. Finally, IBM’s size and influence were essential to the progress and expansion of this effort. Would Magirus have agreed to a project like this with a company that represented only 5% of its business? Would it have been able to convince other distributors to sign on or even have tried to do so? It is not a coincidence that the most often cited B2B examples involve very large companies. They have the power to bring other participants to the table and keep them there long enough to reach Level 2 and Level 3 agreements.

The Future of B2B

All of this brings up a fundamental question: If building application-application interactions between companies is going to continue to be slow and painstaking work, and if the benefits will be slow to appear (and in some cases, difficult to see at all), then why bother, especially since there are so many other uses for managers’ attention, technologists’ time and investors’ dollars? Why not invest only in tools to facilitate human-human and human-application interactions, which are cheaper, much easier and faster to put in place, very flexible and clearly helpful?

Web-services advocates often suggest that anyone asking those questions “just doesn’t get it,” but in many cases unenthusiastic companies actuallydo get it. They’ve evaluated the costs and benefits (both current and potential) of the new technologies and have concluded that they aren’t worth pursuing. The crucial questions remain: When do the new B2B tools make sense, for whom and why?

For application-integration projects within a single company, Web-services technologies usually make great sense. Similarly, intercompany efforts to replace old EDI links — or to build new ones that are so simple that EDI could have handled them — will probably use Web services. Their components are more powerful, more flexible and easier to use, and the Internet is a much cheaper transmission platform than a private EDI value-added network.

The set of EDI links in place today, in fact, gives us a strong indication of where and how the new world of B2B will unfold. EDI exists primarily between large companies and their primary business partners (who are themselves at least midsize) within value chains that are relatively stable. EDI is usually initiated by the most powerful player, which is sometimes a manufacturer, as in the automotive industry, and sometimes a retailer like Wal-Mart Stores Inc. But EDI is rarely established between small companies or between those that expect to do business together only for a short time.

In light of the IBM case study, the reasons for this are clear. B2B links are slow and fairly expensive to set up, inflexible once they’re in place and not immediately expandable to other environments. Furthermore, these links are established in real-world business environments, which are always imposing new requirements on companies over time. Intelligent companies realize upfront that B2B links will need modifications — and will only build them with partners that they trust not to quibble over who pays or to use the project as a negotiating ploy.

Nothing about Web services changes any of this, as the arguments and examples throughout this article are intended to make clear. So we should expect the new B2B technologies to be employed in the same kinds of environments as the old ones. So far, this is exactly what weare seeing. Internetera B2B technologies, including elements of the Web-services tool kit, are being deployed by large companies to accomplish simple transactions and processes. In some cases, they’re being used simply to replace old EDI infrastructure.6

The case of IBM’s midrange B2B ordering effort, however, shows that leading companies are doing more with the new technologies than they could with the old. It therefore tells us something about the future of B2B. As companies get more experience and familiarity with their initial application-application linkages, they will learn which ones are worth expanding, which ones should stay simple and which ones should never have been attempted in the first place. They will advance, one project at a time, into deeper Level 2 and Level 3 interactions with their business partners, and completion of the projects will become quicker and more efficient. These companies may find that XML and the rest of the Web-services tool kit is actuallynot that much different and better than what came before and that it doesn’t make sense to strive for deeper interactions, but that seems unlikely. The Internetera B2B tool kit is a significant advance, and it seems likely that some companies will continue to use it to build richer, broader and deeper links than were previously possible.

Who Should Bother With B2B Web Services?

Does it make sense for you to embark on the slow, difficult journey toward deep application-application interaction with your business partners? And, if so, what’s the right way to go about it? To answer the first two questions, ignore the publicity and the predictions about Web services and B2B for a moment and think about what your business needs, rather than what new tools are available. Would linking your applications to another company’s be highly valuable? Can you quickly list the first set of transmissions and business processes you’d execute, the next set and so on? Can you identify the benefits, advantages and new capabilities that could (eventually) come from this kind of automation? And are these benefits particularly compelling, given your competitive situation? If a rival acquired them and you didn’t, would you be in a great deal of trouble?

Finally, are the benefits from application-application interaction clearly greater than those resulting from human-human or human-application links? Recall that before beginning the B2B effort, Magirus employees weren’t faxing or phoning in their orders; they were typing them directly into IBM’s fulfillment systems via a Web site. Human-application interactions, in other words, were already occurring between the two companies; the B2B effort simply replaced some of them with application-application ones. Not every company, however, should make the same choice that Magirus and IBM did. When interactions between companies are less frequent and/or less important, it probably doesn’t make sense to replace manual links with automatic ones.

If your answers are mostly “no,” it’s probably not worthwhile to pursue application-application interactions. If you answered yes a number of times, it makes sense to explore the new world of B2B. But it’s important not toassume a yes answer to any of the above questions; only internal analysis will reveal whether B2B makes sense. Building application-application links between companies is not a universal competitive imperative. Like any other initiative, it makes sense in some circumstances and not in others.

If you embark on B2B initiatives, set and manage expectations properly. Don’t expect, for example, that the companies in your industry are going to set and adhere to Level 2 and Level 3 standards. And regardless of the state of the industry-level standards, the household problem, specialized business requirements (as seen in the IBM case) and the quest for competitive advantage will combine to ensure that company-level B2B e-documents and business processes will always be highly fragmented in nature. Similarly, don’t expect that all of the partners you want to work with will be equally excited about B2B or will be immediately willing to adopt the Level 2 and Level 3 solutions you propose. IBM’s work with its business partners, like many other similar projects, also shows the importance of working with trusted partners, leveraging the work of consortia and writing B2B “cookbooks” so that later projects ramp up faster and more easily than initial ones.

Most fundamentally, the other B2B efforts I’ve studied show that the process is more like erosion than an earthquake. It does change the landscape, but slowly and via many small events instead of one big one. XML, Web services and B2B truly are large and sudden leaps forward in technology. But this does not mean that they will cause equally large and quick changes in the business world. Because of the amount of upfront learning and negotiation required to reach Level 2 and Level 3 agreements, application-application links will not propagate spontaneously or quickly. They will appear one by one, thanks to the dedicated work of managers, leaders and teams.

In this sense, then, B2B technologies are much more like intra-company enterprise applications such as enterprise resource planning (ERP) than they are like intercompany innovations such as e-mail and the Web. Both B2B and ERP are technologies that, to be successful, require skillful change management over a long period of time.7 Companies and managers that expect a quick win or quick impact from them are going to be disappointed. So are those who adopt them because everyone else is or because they’re the latest must-have technology. Adopters with clear and reasonable expectations, patience and effective programs for executing IT-enabled change, on the other hand, will eventually reap the benefits of B2B technologies.

The Impact on Competing and Collaborating

A better understanding of the work required to construct application-application links sheds light on two important topics: whether the new B2B technologies can be used to build and sustain competitive advantage for a company, and how they will affect collaboration networks.

Many authors have correctly pointed out that for any IT to be a source of sustained competitive advantage, it must be both valuableand difficult to acquire or replicate. Some have concluded from this reasoning that information technologies will meet this test less and less often, since they’re continually becoming cheaper and more standardized.8 While it’s true that theraw materials of information technology — hardware, networks and commercially available software — are becoming very widely available, this does not mean that they can always be easily converted into thefinished goods of IT — technologies that are being used productively and adding value. The household problem, the difficulty of solving it and the IBM case study all indicate that B2B is not easy now and is not going to get much easier, no matter how good the technology tool kit becomes.

As a result, B2B links will likely be viable sources of competitive advantage in many circumstances. As discussed above, B2B will not be valuable or appropriate for every company at every point in time. In situations where it is valuable, it will also be a differentiator, because a B2B infrastructure can’t simply be installed; it must be built and managed over an extended period of time. It is organizationally difficult, and success is far from guaranteed, but companies that do succeed with it find themselves with a competitively valuable capability.

The fact that the main difficulties with B2B are organizational, rather than technical, has an important implication. It means that Web services are very unlikely to lead to the “unbundling of the corporation” into modular organizations that combine and recombine themselves into virtual companies. There is clear evidence that IT is associated with greater out-sourcing and that the technologies of human-human and human-application interaction are allowing work to migrate around the world.9 However, application-application links are present only in the most stable of these collaborations, where it is clear who (usually the bigger, more powerful partner) gets to make what decisions about the link. These collaborations, in other words, do not operate as dynamic spot markets; they are much more like traditional hierarchies.

When Internet-era B2B technologies first appeared, many people, myself included, expected them to quickly lead to dynamic and liquid electronic markets composed of specialized companies.10 Real-world considerations and real-world experience, however, have intruded on this vision. It turns out that, in addition to excellent technologists, skilled managers and leaders are needed to build application-application links. It also turns out that a bit of market power goes a long way in convincing partners to share your B2B vision.

Long-term collaborations involving large companies often meet these requirements; short-term transactions conducted in the marketplace rarely do. Because of this, B2B will happen within electronic hierarchies, not across electronic markets. Hard-wired partnerships like the ones between IBM and its distributors will become more common but will never become trivial to build. We should expect to see the construction of these kinds of partnerships become one of the most important IT-based competitive battlegrounds in the coming years. If you think that they’ll be important in your industry, don’t expect them to form themselves. Start forming them now.