How Developers Can Lower AI’s Climate Impact

AI is booming. The public release of large language models like ChatGPT has popularized the technology, which was already becoming a critical driver of companies’ efforts to innovate and grow. But as these models get bigger, so too does their appetite for energy: Training the open multilingual language model BLOOM produced nearly 24.7 tons of carbon emissions. AI itself might be a valuable tool for helping to find opportunities for sustainability improvements, but it could also become a drag on collective efforts to mitigate the global climate emergency.

Managers know that accurate metrics are the starting point for getting a handle on any problem, but it’s not easy to estimate the energy consumption of AI and machine learning (ML) models. Most AI companies don’t measure and disclose this parameter. Energy consumed during deployment is even less well understood than consumption during training.

There are tools available to help. The Software Carbon Intensity specification from the Green Software Foundation outlines a reliable approach for determining a carbon emissions baseline that can then be used for comparison over time or across applications. The Green Algorithms project offers a simple calculator to estimate the total emissions of an AI project. Amazon Web Services, Google Cloud Platform, and Microsoft Azure offer carbon accounting tools specific to their cloud services. Researchers at Stanford, working with industry stakeholders, have published a lightweight framework for reliable, simple, and precise reporting of the energy, compute, and carbon impacts of machine learning systems.

Taking a Life-Cycle Approach to Mitigation

While measurement can reveal the status quo and help organizations track their progress on efforts to improve, actually moving the needle on AI-related carbon emissions requires addressing each step of the development, implementation, and adoption life cycle.

Different frameworks are emerging to meet this need. A joint study by Google and the University of California, Berkeley demonstrated that the energy consumption of ML training can be lowered up to 100x and CO₂ emissions up to 1,000x by applying four best practices: selecting efficient ML model architectures, such as sparse models versus dense models; using processors optimized for ML training, such as tensor processing units (TPUs) or recent graphics processing units (GPUs) rather than general-purpose processors; using cloud computing resources for better data center efficiency; and choosing cloud service locations that use clean energy.

We recommend an approach that emphasizes three actions: Rethink, reuse, and reduce. Applied across the full AI life cycle, these actions can generate more value from less data, cut costs, and reduce AI’s environmental impact.

Rethink

Begin by asking the question that is becoming ever more salient in AI research: Is more data always better than less? In an experiment with a Kaggle data set of nature scenes, we found that training a computer vision model on 70% of the full data set reduced its accuracy by less than 1% but cut energy consumption by 47%. It also revealed that one-time, big-bang data learning consumes less energy than continuous learning. And retraining the model makes sense only in specific situations, such as if the amount of training data crosses a certain threshold.

When collaborating with other organizations, federated machine learning allows a single model to be trained on data from multiple sources (without explicitly sharing the data). This is done to improve model quality but also yields substantial cost and energy savings by reducing the amount of data required to be generated and stored by each organization.

Also, the quality of data matters as much as the quantity. Using class-imbalance training data sets — where observations related to one class, such as dark-skinned Asian females, are far outnumbered by another class, such as white Caucasian males — consumed three times the energy and required double the data to achieve the same level of accuracy as training with a class-balanced data set. Ensuring that there are no or very few data quality issues will help you reach the minimum performance requirements (such as accuracy and precision) faster, with less energy consumption.

Deep learning requires huge amounts of data and is highly compute- and power-intensive. Pruning these models by removing unused parameters can help decrease model size, increase inference speed, and reduce the need for multiple iterations, thus resulting in energy savings. An alternative approach to deep learning is sparse modeling, which can be highly effective in creating embedded low-power applications using limited data sets.

In general, businesses should pause to consider which is the most appropriate AI technique for the task at hand. Generative AI, which can also consume the most energy, is most applicable to coding software, creating novel content, and automating discovery. For many applications, a simpler predictive AI technique will be more appropriate — and greener — than a generative AI approach.

Reuse

Techniques such as transfer learning allow a model trained on one task to be repurposed for a related task, reducing costs and emissions related to computation and data storage. We found that when training a computer vision model, the transfer learning approach took 22 fewer epochs (one pass of the entire training data through the algorithm) and was four times faster in reaching the same level of accuracy (95%) as training a model from scratch. It also consumed 99.3% less energy.

Different transfer learning techniques can have different impacts on the amount of energy consumed by language models. Pretraining the model on specific tasks (such as text classification or language translation) with a reduced data set consumes less energy than when the model is pretrained with a full-domain data set (such as medical or legal text). Task-adaptive pretraining also results in a lower perplexity score, a measure of how well the model can predict the next word in a sentence.

During deployment, it will likely pay to distill the knowledge from a large teacher model (large models are heavily used for inference) into a smaller, student model. In our own experiment, a student model consumed 2.7 times less energy during inference than the original teacher model. However, this approach might not work well if the original model requires frequent retraining, because the smaller student model would also need to be retrained.

Reduce

There are also promising approaches emerging to reduce the energy consumption of the GPUs typically used in the model training process. Researchers at Harbin Institute of Technology and the University of Leeds have developed GPOEO, a micro-intrusive GPU online energy optimization framework that can automatically adjust the energy settings of GPUs based on the type and pattern of the tasks they are performing. This approach is fairly easy to apply, given that only the start and end points of the code section that is to be optimized need to be marked. GPOEO then monitors the program behavior, detects the workload patterns, and finds the best energy configuration for each pattern. The researchers claim that GPOEO can significantly reduce the energy consumption of GPUs without compromising their performance.

Once a model is deployed, hardware infrastructure choices can also yield energy savings, with GPUs and TPUs outperforming CPUs. It is fairly easy to make these choices because many cloud providers offer compute on demand that is better optimized for energy efficiency. Further, in an in-house exploration, we found that upgrading to the latest versions of virtual machines provided in the cloud can cut energy use and emissions by half, for the same cost or sometimes even less.

---

If AI is to be a net positive for the planet and society, we must rein in the amount of energy it consumes and the corresponding carbon emissions. Rather than racing recklessly into AI without considering the consequences for their organization’s carbon footprint, leaders must gain an awareness of the actual energy consumption of AI initiatives and make mitigation a key objective of development and deployment.