ChatGPT Needs a Chemistry Tutor Too

Student performance on open-response calculations involving acid and base solutions before and after instruction in general chemistry and analytical chemistry courses was compared with the output from the artificial intelligence chatbot ChatGPT. Applying a theoretical model of expertise for problem solving that includes problem conceptualization, problem strategy, and solution, it is found that students errors following instruction primarily involved problem conceptualization and the misapplication of heuristics like the Henderson−Hasselbalch equation When the same problems were used as input to ChatGPT the responses were comparable to worked examples found in general chemistry textbooks in terms of length and detail and usually displayed strong problem conceptualization. Response accuracy of the chatbot varied greatly for different topics, being best for calculations of pH for a strong acid or strong base and much lower for more complex problems involving titrations or aqueous salts. Chatbot and student errors differed in that the chatbot did not misapply heuristics but did make mathematical errors uncommon for students. The variability in the correctness of ChatGPT's responses and the nature of its errors vis-a-vis students will influence its potential use as an instructional resource for calculations involving acids and bases.

Section: Introductionmentioning

confidence: 95%

Section: Introductionmentioning

confidence: 99%

Comparing the Performance of College Chemistry Students with ChatGPT for Calculations Involving Acids and Bases

Clark,

Anderson,

Dickson-Karn

et al. 2023

“…Also, the usefulness of prompting frameworks and the students' use of text-generative AI are described in this context. The work does not aim to verify whether the answers given by AI were correct, as this has been considered many times, 31,32,47 and it is expected that students will tend to rate the answers to be more correct than experts would. 35 Altogether, the main aim of this study is to develop a deeper understanding of prompting skills and to investigate how to implement the Five S framework to develop those skills.…”

Section: Aim Of This Studymentioning

confidence: 99%

“…Furthermore, chemistry questions in multiple-choice and open format of a general chemistry exam were answered with low accuracy. 31,32 As for students, it was shown that AI tools are used by two-thirds of German university students, 33 but they have mediocre AI literacy. 34 There even are reports of unreflected acceptance of ChatGPT's answers when solving physics tasks.…”

Section: ■ Introductionmentioning

confidence: 99%

Assessment of Students Use of Generative Artificial Intelligence: Prompting Strategies and Prompt Engineering in Chemistry Education

Tassoti

2024

The rapid integration of generative artificial intelligence (AI) into educational settings prompts an urgent examination of its efficacy and the strategies that students employ to harness its potential. This study focuses on preservice chemistry teachers use of generative AI for chemistry-specific problem-solving and task completion. We found that there is a prevalent reliance on copy-pasting tactics in initial prompting approaches, and students need guidance to improve their prompting abilities. By implementing the “Five S” prompting framework, we explore the evolution of student prompts and the resultant satisfaction with AI-generated responses. Our findings indicate that, while students initially struggle with the nuances of effective prompting, the adoption of structured frameworks significantly enhances their perceived quality of AI-generated answers. This research sheds light on the current state of AI use among students but also underscores the importance of targeted educational frameworks to refine AI interaction in academic contexts. In particular, we suggest critical engagement and methodological prompt engineering strategies to maximize the educational benefits of generative AI technologies.

“…An early investigation using ChatGPT-3.5 to answer chemistry questions found that the chatbot had strong natural language processing skills (NLP) to communicate information, but that the accuracy of the chatbot’s output was a concern . Shortcomings involving the accuracy of ChatGPT-3.5 have since been discussed in early stage reviews of the use of Artificial Intelligence (AI) in educational settings, − leading some to conclude that inaccuracies are a serious impediment to its use. , Others have looked closely at the nature of these errors and considered them in relation to student reasoning and problem-solving. Fergus and co-workers found that ChatGPT’s answering of questions involving the interpretation of nontext information and the application of knowledge were weaknesses for the chatbot .…”

Section: Introductionmentioning

confidence: 99%

Using ChatGPT to Support Lesson Planning for the Historical Experiments of Thomson, Millikan, and Rutherford

Clark,

Fhaner,

Stoltzfus

et al. 2024

Four General Chemistry instructors investigated the use of ChatGPT-4 to improve their lessons plans for the historical experiments of Thomson, Millikan, and Rutherford. The instructors varied in their prior knowledge for these experiments and their initial lessons addressed somewhat different learning objectives. This led to different conversations with the chatbot as the instructors used the resource in different ways and discussed topics they each found relevant. The output from ChatGPT-4 was robust and each instructor identified ways it could be used to improve their instruction. The chatbot was able to accomplish instructional tasks these instructors found useful, such as outlining a lesson plan, recommending resources, discussing instructional strategies, describing calculations, offering explanations for different levels of leaners, and generating assessments. A limitation was its ability create images or visual aids the instructors found useful. Overall, these instructors found the chatbot could support, but not replace, an instructor in a course like General Chemistry.