ChatGPT Convincingly Explains Organic Chemistry Reaction Mechanisms Slightly Inaccurately with High Levels of Explanation Sophistication

Abstract

The chemistry education research community values and emphasizes the role of constructing explanations and mechanistic reasoning to support students' learning of organic chemistry. Emerging large language model (LLM) and generative artificial intelligence (GAI) technologies are uniquely equipped to advance the teaching and learning of chemistry. GAI-based chatbots, such as ChatGPT, have the potential to help students learn mechanistic reasoning through their generated responses. This study investigates the extent to which 255 ChatGPTgenerated responses are accurate explanations of 85 different reaction mechanisms and exhibit mechanistic reasoning as categorized by the levels of explanation sophistication framework. The study also explores the effects of prompt engineering on mechanism accuracy and explanation sophistication through three types of prompt cueing. Study findings show that (1) a quarter of responses are fully accurate explanations of reaction mechanisms and the majority contain predominantly accurate explanations of chemical phenomena and identification of nucleophiles and electrophiles, (2) responses exhibit high levels of explanation sophistication, and (3) prompt engineering plays a significant role in eliciting high levels of explanation sophistication but not mechanism description accuracy. Results are situated in mechanistic reasoning and prompt engineering frameworks with a focus on how these new technologies can be integrated into the chemistry classroom.

Using ChatGPT-4 to Teach the Design of Data Visualizations

“…In the Fall of 2023, my course used one of these AI tools (ChatGPT-4) as the predominant means for creating and refining data visualizations, and this manuscript discusses the approach used as well as its relative merits and limitations as well as student perceptions and outcomes. It should be mentioned that there is already significant discussion of the role of AI tools in chemical education, focusing on topics such as writing aids, lesson planning aids, , strengths and shortcomings for problem solving, − concerns over academic integrity, and enhancing student learning activities. ,, However, to my knowledge, there are yet no articles discussing the use of AI to help teach the design of data visualizations. It is hoped the approach discussed below could be incorporated into other courses, thereby better preparing students to effectively communicate their science.…”