The Problem With AI Pilots

Over the past year or so we’ve been engaged in an effort to tell the story of how large organizations are deploying artificial intelligence in their businesses. We were encouraged by the response to the 2018 NewVantage Partners executive survey, in which 93% of respondents said their organizations were investing in AI initiatives. Plenty of companies to write about, we thought. These were very large organizations spending goodly sums on AI and with a history of early adoption of other technologies. But when we approached many of these companies to discuss writing some case studies about their work, most of them demurred.

Most said the reason wasn’t that they wanted to keep their AI activities secret, but that they weren’t actually very far along and hence their projects were not worth discussing yet. They were doing lots of pilots, proofs of concept, and prototypes, but they had few production deployments. When they did have AI systems in production, most were machine learning-based systems that had been in place for many years. This is particularly true in financial services, where large-scale “scoring” has been used to evaluate customers for credit and potential fraud for well over a decade. Some said to us that they didn’t really consider these projects to be examples of AI — consistent with the common view of AI that it describes technology that is never really here yet. Others say that they have robotic process automation (RPA) implementations in place, but most are relatively small, and there is also debate about whether RPA is really AI or not.

Why AI Implementation Is Challenging

But there are good reasons why production implementations of AI technology are relatively scarce. One is the maturity — or lack thereof — of the technology. Chatbots and intelligent agents, for example, are getting better all the time, but many companies still hesitate to turn their customers over to them. Instead, they ask their employees to use them for applications in HR and IT. Some make them available to their call center reps to use in the background to help answer customer questions. Eventually, they hope, they will support customer interactions directly.

If the AI initiative actually changes the relevant business process and the skills necessary to perform it, that raises another barrier to full implementation — the old bugaboo “change management.” Most AI systems still involve some interaction with human workers, and educating those workers on new tasks and new skills can be time-consuming and expensive. UPS, for example, developed a complex machine learning algorithm for daily routing of its package delivery trucks, and it’s still rolling it out 10 years after the algorithm was developed. Getting tens of thousands of drivers to change their behavior isn’t easy. Similarly, a new claims process in an insurance company we worked with involves using deep learning models to analyze photos of car accidents. The technology works pretty well, but it doesn’t work with all types of collision damage. The interface of the AI system with existing claims adjusters, who are still needed for most damage assessments, and their existing work processes has been challenging.

Full production implementation also involves interfacing AI with production information systems and architectures. A 2017 Deloitte survey found that the number one obstacle to successful AI deployments was that it was “difficult to integrate cognitive projects with existing processes and systems.” New machine learning models may have to be written as APIs or as program code modules within existing systems. Even RPA systems, which are quite easy to implement in small volumes, can become an architectural challenge when adopted in large numbers. Because they act as users of production systems, they are typically impacted by changes in those systems and may have to be reprogrammed.

Making AI Implementation More Likely

There are several ways that companies can increase the likelihood and speed of production AI implementations. Here are some of them:

• Set a time and criteria for deciding whether to go into production before the pilot starts. This will add rigor to the decision process and put pilot project advocates on notice that implementation is an important consideration from the beginning (this should help to address all the challenges mentioned above).
• Adopt technologies that can scale and that can be used by the intended audience. If, for example, it is unlikely that a chatbot will be made available to customers as a primary channel, don’t adopt it with vague hopes that it will improve its performance quickly (helps with the technological maturity challenge).
• Adopt AI capabilities that are already embedded within transaction systems. Major transaction system vendors, including Salesforce, SAP, Oracle, and Workday, are adding AI capabilities to their offerings. That typically means that the AI offerings will be somewhat integrated with transaction systems from the beginning, and that they can make use of the data within those systems (helps with the integration challenge).
• If the AI system will be stand-alone, make sure it can create a relatively easy interface with your existing systems — such as an API or generated program code that works with your architecture (also assists in integration).
• Start an AI-related education and skills program now. Even though you may not be sure of the specific needs of your workers for retraining and re-skilling, you can make available education offerings now about how to understand and work with smart machines. Such programs put employees on notice that change in their jobs from AI will happen and that they should begin preparing for it (and helps with the change management challenge).

Implementation of AI projects is the only way that organizations will realize tangible business value from their AI investments. Companies are spending considerable sums on AI technology, and it should not be viewed exclusively as an experiment. It is only when companies step up to production status with AI that it will deliver ROI and productivity for their organizations.