Beware the Paradox of Automation

The paradox of automation: Earlier this year, Facebook exorcised those pesky human editors who were introducing political bias into its Trending news list and left the job to algorithms. Now, reports Caitlin Dewey in The Washington Post, the Trending news isn’t biased, but some of it is fake. Turns out the algorithms can’t tell a real news story from a hoax.

Facebook says it can improve its algorithms, but errors of judgment aren’t the only pitfall in transferring human tasks to machines. There’s also the paradox of automation. “It applies in a wide variety of contexts, from the operators of nuclear power stations to the crews of cruise ships, from the simple fact that we can no longer remember phone numbers because we have them all stored in our mobile phones, to the way we now struggle with mental arithmetic because we are surrounded by electronic calculators,” says Tim Hartford in an excerpt published by The Guardian from his new book, Messy: The Power of Disorder to Transform Our Lives. “The better the automatic systems, the more out-of-practice human operators will be, and the more extreme the situations they will have to face.”

Hartford borrows William Langewiesche’s harrowing description of the crash of Air France Flight 447 to illustrate three problems with automation: “First, automatic systems accommodate incompetence by being easy to operate and by automatically correcting mistakes. … Second, even if operators are expert, automatic systems erode their skills by removing the need for practice. Third, automatic systems tend to fail either in unusual situations or in ways that produce unusual situations, requiring a particularly skillful response.”

The excerpt is worth a read — especially if it prompts you to ask if your company’s automation initiatives might entail similar risks.

AI stimulus from the U.S. government: In August, this column included a link to IBM’s response to a call from the White House for information on artificial intelligence. Last week, the White House issued its own AI report and a strategic framework for developing a national AI capability. And U.S. commander in chief Barack Obama discussed it, along with MIT Media Lab director (in chief) Joi Ito, in an extensive interview with Wired editor in chief Scott Dadich.

There’s news and possibly opportunities in all this for companies that are pursuing AI-related initiatives.