Comparative Analysis of Generative Pre-Trained Transformers Responses to Coastal Ecosystem Questions: Implications for GenAI in Environmental Education

Abstract

This study presents a comparative analysis of two leading large language models (LLMs), Google’s Bard, powered by the Gemini model, and OpenAI’s ChatGPT 4.0, in the context of their responses to coastal ecosystem science undergraduate student education. Fifty questions related to coastal ecosystem management were posed to each LLM. Expert assessments evaluated the responses of the LLMs based on five key metrics: accuracy, relevance, depth, creativity, and semantic clarity. Knowledge graphs provided a structured framework for assessing and visualizing the AI responses. The analysis identified the strengths and weaknesses of each LLM in addressing complex environmental issues. The findings contribute to a deeper understanding of LLMs’ potential applications in environmental science education and scientific communication. This study acknowledges limitations, such as the inherent subjectivity of expert assessments and the potential for bias within the knowledge graphs used for evaluation. Future research directions include investigating the effectiveness of LLMs in personalized learning environments and exploring their potential for generating educational content tailored to diverse audiences.