ChatClimate: Grounding Conversational AI in Climate Science

Abstract

Large Language Models (LLMs) have made significant progress in recent years, achieving remarkable results in question-answering tasks (QA). However, they still face two major challenges at inference time: hallucination and outdated information. These challenges take center stage in critical domains like climate change, where obtaining accurate and up-to-date information from reliable sources in a timely fashion is essential. To overcome these barriers, one potential solution is to provide LLMs with access to external, scientifically accurate, and robust sources (long-term memory). This access assists LLMs in continuously updating their knowledge and preventing the propagation of inaccurate, incorrect, or outdated information. In this study, we enhanced GPT-4 by providing it access to the Sixth Assessment Report of the Intergovernmental (IPCC-AR6), the most comprehensive, up-to-date, and reliable source in this domain. We present our conversational AI prototype, available at www.chatclimate.ai. The product answers challenging questions accurately in three different QA scenarios: asking from 1) GPT-4, 2) ChatClimate, and 3) hybrid ChatClimate. The answers and their sources were evaluated by a team of IPCC authors, who used their expert knowledge to score the accuracy of the answers from 1 (very-low) to 5 (very-high). The evaluation showed that the Standalone assistant provided more accurate answers.