Predictive Models Can Lose the Plot. Here’s How to Keep Them on Track.

Organizations are increasingly turning to sophisticated data analytics algorithms to support real-time decision-making in dynamic environments. However, these organizational efforts often fail — sometimes with spectacular consequences.

In 2018, real estate marketplace Zillow launched Zillow Offers, an “instant buyer” arm of the business. It leveraged a proprietary algorithm called Zestimate, which calculates how much a given residential property can be expected to sell for. Based on these calculations, Zillow Offers planned to purchase, renovate, and resell properties for a profit.1 While it had some success for the first few years, the model failed to adjust to the new dynamics of a more volatile market in 2021. Zillow lost an average of $25,000 on every home it sold in the fourth quarter that year — resulting in a write-down of $881 million.2

This is an example of what we call algorithmic inertia: when organizations use algorithmic models to take environmental changes into account but fail to keep pace with those changes. In this article, we explain algorithmic inertia, identify its sources, and suggest practices organizations can implement to overcome it.

A Credit-Rating Catastrophe

To understand the phenomenon of algorithmic inertia, we conducted an in-depth study of another organization that failed to respond to changes in the environment: Moody’s, a financial research firm that provides credit ratings for bonds and complex financial instruments such as residential mortgage-backed securities (RMBSs). These securities aggregated bundles of individual mortgages into distinct tranches with unique characteristics during the period leading up to the global financial crisis of 2008.