Towards Climate Awareness in NLP Research

Abstract

The climate impact of AI, and NLP research in particular, has become a serious issue given the enormous amount of energy that is increasingly being used for training and running computational models. Consequently, increasing focus is placed on efficient NLP. However, this important initiative lacks simple guidelines that would allow for systematic climate reporting of NLP research. We argue that this deficiency is one of the reasons why very few publications in NLP report key figures that would allow a more thorough examination of environmental impact. As a remedy, we propose a climate performance model card with the primary purpose of being practically usable with only limited information about experiments and the underlying computer hardware. We describe why this step is essential to increase awareness about the environmental impact of NLP research and, thereby, paving the way for more thorough discussions.

Enhancing Large Language Models with Climate Resources

“…Carbon Footprint. While LLMs have demonstrated impressive capabilities in generating human-like text across diverse topics, they come with a significant energy consumption cost (Hershcovich et al 2022). Training these models requires massive amounts of computational resources, which in turn generates considerable greenhouse gas emissions.…”

Evaluating text classification: A benchmark study

“…However, such methods require substantially more computational resources compared to simpler 1 methods. Additionally, with growing concerns regarding the excessive use of computational resources (Ulmer et al, 2022;Hershcovich et al, 2022;Bannour et al, 2021) and their ecological footprint, it is imperative to consider computationally efficient methods, especially when they are not necessary for the task.…”

Fine-tuning ClimateBert transformer with ClimaText for the disclosure analysis of climate-related financial risks

“…Since its publication in October 2021, the pretained transformer model Cli-mateBert has only been applied so far in 6 other papers which have not been published yet (two of which are from ClimateBert's original authors): [17,4,30,7,13,36]. Hershcovich et al [17] analyze ClimateBert only regarding its energy consumption, in a context of awareness about the environmental impact that NLP pretrained models present. Focusing on the policies being adopted by governments around the world, Sietsma et al [30] identify ClimateBert as a tool that allows real-time tracking of adaptation progress.…”