Thinking critically about scientific information generated by ChatGPT

Abstract

Purpose In November 2022, the commercial company, OpenAI, launched ChatGPT. Since then, university students have rapidly become regular users of this artificial intelligence (AI) platform. One reason for this is the powerful capability of this generative AI tool to produce textual content, which in many cases, is almost indistinguishable from human-generated content. Another reason is that ChatGPT easily gives anyone access to knowledge. However, there is a problem as the vast majority of its users have no idea how this AI platform works and thus overlook the importance of thinking critically about information communicated in ChatGPT. While some call for banning this generative AI tool, this study aims to provide evidence that science classrooms can become scenarios where students find explicit, concrete, and realistic opportunities to critically evaluate scientific information generated by ChatGPT. Design/methodology/approach An intervention study was conducted with 55 students (26 females and 29 males, 17–24 years old) during a university Spanish-English bilingual science course taught within an active learning environment. The data consist of the written critiques of the students about Spanish-English bilingual scientific texts produced by ChatGPT. Findings Results indicate that the intervention had a positive effect on students’ abilities to construct sound arguments in Spanish and in English while judging the quality of scientific texts produced by this AI bot. Moreover, the findings suggest that the intervention enriched students’ skills to make improvements to texts produced by this generative AI tool. Originality/value The evidence provided in this study contributes to the exploration of possibilities to help students become critical users of ChatGPT.