ChatGPT—A double‐edged sword for healthcare education? Implications for assessments of dental students

Abstract

IntroductionOpen‐source generative artificial intelligence (AI) applications are fast‐transforming access to information and allow students to prepare assignments and offer quite accurate responses to a wide range of exam questions which are routinely used in assessments of students across the board including undergraduate dental students. This study aims to evaluate the performance of Chat Generative Pre‐trained Transformer (ChatGPT), a generative AI‐based application, on a wide range of assessments used in contemporary healthcare education and discusses the implications for undergraduate dental education.Materials and MethodsThis was an exploratory study investigating the accuracy of ChatGPT to attempt a range of recognised assessments in healthcare education curricula. A total of 50 independent items encompassing 50 different learning outcomes (n = 10 per item) were developed by the research team. These included 10 separate items based on each of the five commonly used question formats including multiple‐choice questions (MCQs); short‐answer questions (SAQs); short essay questions (SEQs); single true/false questions; and fill in the blanks items. Chat GPT was used to attempt each of these 50 questions. In addition, ChatGPT was used to generate reflective reports based on multisource feedback; research methodology; and critical appraisal of the literature.ResultsChatGPT application provided accurate responses to majority of knowledge‐based assessments based on MCQs, SAQs, SEQs, true/false and fill in the blanks items. However, it was only able to answer text‐based questions and did not allow processing of questions based on images. Responses generated to written assignments were also satisfactory apart from those for critical appraisal of literature. Word count was the key limitation observed in outputs generated by the free version of ChatGPT.ConclusionNotwithstanding their current limitations, generative AI‐based applications have the potential to revolutionise virtual learning. Instead of treating it as a threat, healthcare educators need to adapt teaching and assessments in medical and dental education to the benefits of the learners while mitigating against dishonest use of AI‐based technology.

Performance of ChatGPT on Registered Nurse License Exam in Taiwan: A Descriptive Study

“…The current investigation utilized a multiple-choice examination format. ChatGPT demonstrated the capability to furnish rapid responses within seconds, as well as to provide answers and information during the interaction, consistent with prior research by Ahn (2023) [9] and Ali et al (2023) [18]. The findings unveiled that ChatGPT possessed a rudimentary grasp of nursing knowledge, resulting in a success rate exceeding 54.05%.…”

“…ChatGPT's medical knowledge and interpretative capabilities within the field of medicine remained insufficient and unreliable. Ahn (2023) [10] highlighted ChatGPT's tendency to retrieve erroneous information and skills through web searches, potentially stemming from inaccurate data sources. According to the findings of this study, ChatGPT exhibited the potential to generate misleading or incorrect explanations, even though it adeptly rephrased unclear or incomprehensible questions [2].…”

“…The utility of ChatGPT encompasses its role as an interactive tool for medical education to facilitate the learning process [1,3]. Moreover, it has the potential to revolutionize the virtual learning of students [10]. However, the application of ChatGPT has its constraints, which encompass the possibility of generating biased output, concerns regarding information Healthcare 2023, 11, 2855 2 of 12 accuracy, and its potential influence on students' critical thinking and communication skills, among other factors [7,11,12].…”

“…All these opinions show that we need to apply evidence-based research to understand the potential shortcomings of implementing ChatGPT in healthcare education [12]. AI chatbots still lack the ability to present statistical analyses, and although they can simulate text for various scenarios, they are unable to generate visual animations [2,10,24].…”

ChatGPT to enhance learning in dental education at a historically black medical college

“…To fully realize the potential bene ts of LLMs for dental education, the limitations and accuracy of LLMs needs to be assessed, so that effective quality control can be designed and implemented to safeguard against inaccurate and biased responses to dental-related queries [10]. Despite many analyses conducted to assess LLMs' responses [11][12][13], the research in this area is still in its infancy, and the implications of LLMs for dental education remain insu ciently understood. In this paper, we explore the utility and effectiveness of LLMs to enhance dental students' learning experiences, with a focus on the use of ChatGPT [1], one of the leading LLMs.…”