The Role of Artificial Intelligence in Higher Education: ChatGPT Assessment for Anatomy Course

Talan, Tarık; Kalinkara, Yusuf

doi:10.33461/uybisbbd.1244777

Cited by 52 publications

(34 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3,4 ChatGPT's performance, with a success rate of 67.5%, was better than the average performance of students enrolled in an anatomy course at the Faculty of Health Sciences. 5 In addition, it performed well in typical clinical toxicology cases of acute organophosphate poisoning 6 and also in a multiplechoice exam for the course of Family Medicine in the third year of medical school at Antwerp University, Belgium. 7 ChatGPT has progressed rapidly over time, even passing the American Board of Neurological Surgery Self-Assessment Exam.…”

mentioning

confidence: 97%

ChatGPT failed Taiwan’s Family Medicine Board Exam

Weng

Wang

Chang

et al. 2023

Journal of the Chinese Medical Association

View full text Add to dashboard Cite

Background: Chat Generative Pre-trained Transformer (ChatGPT), OpenAI Limited Partnership, San Francisco, CA, USA is an artificial intelligence language model gaining popularity because of its large database and ability to interpret and respond to various queries. Although it has been tested by researchers in different fields, its performance varies depending on the domain. We aimed to further test its ability in the medical field. Methods: We used questions from Taiwan’s 2022 Family Medicine Board Exam, which combined both Chinese and English and covered various question types, including reverse questions and multiple-choice questions, and mainly focused on general medical knowledge. We pasted each question into ChatGPT and recorded its response, comparing it to the correct answer provided by the exam board. We used SAS 9.4 (Cary, North Carolina, USA) and Excel to calculate the accuracy rates for each question type. Results: ChatGPT answered 52 questions out of 125 correctly, with an accuracy rate of 41.6%. The questions’ length did not affect the accuracy rates. These were 45.5%, 33.3%, 58.3%, 50.0%, and 43.5% for negative-phrase questions, multiple-choice questions, mutually exclusive options, case scenario questions, and Taiwan’s local policy-related questions, with no statistical difference observed. Conclusion: ChatGPT’s accuracy rate was not good enough for Taiwan’s Family Medicine Board Exam. Possible reasons include the difficulty level of the specialist exam and the relatively weak database of traditional Chinese language resources. However, ChatGPT performed acceptably in negative-phrase questions, mutually exclusive questions, and case scenario questions, and it can be a helpful tool for learning and exam preparation. Future research can explore ways to improve ChatGPT’s accuracy rate for specialized exams and other domains.

show abstract

mentioning

confidence: 97%

ChatGPT failed Taiwan’s Family Medicine Board Exam

Weng

Wang

Chang

et al. 2023

Journal of the Chinese Medical Association

View full text Add to dashboard Cite

show abstract

“…Talan & Kalinkara, Y [ 42 ]. compared the performance of ChatGPT in an anatomy course with that of undergraduate students.…”

Section: Application and Perception Of Chatgpt In Higher Educationmentioning

confidence: 99%

Understanding learners’ perceptions of ChatGPT: A thematic analysis of peer interviews among undergraduates and postgraduates in China

Xu,

Su,

Zhang

et al. 2024

Heliyon

View full text Add to dashboard Cite

“…However, empirical exploration is needed to understand its implications fully (Compton & Burke, 2023). ChatGPT's ability to emulate human composition poses challenges in detecting authenticity and emphasizes the importance of ethical considerations in its application (Talan & Kalinkara, 2023).…”

Section: Leveraging Ai-powered Technology For Enhanced Accuracy and R...mentioning

confidence: 99%

The Paradigm Shift in Education in Big Data Era: Exploring the Intersection of Historical Assessment Frameworks and AI-Powered Assessment Methods in Education

Du,

Geiger,

Killen

et al. 2024

JETS

View full text Add to dashboard Cite

This paper delves into the dynamic intersection between traditional assessment processes and AI-powered technology assessment models within the expansive landscape of the big data era. Tracing the origins of large-scale standardized testing, from Francis Galton's pioneering intelligence testing proposals to the subsequent development of norm-referenced testing (NRT) by Alfred Binet and Theodore Simon, the historical trajectory underscores the evolution towards authentic assessment paradigms such as computerized adaptive testing (CAT) and performance-based assessment (PBA). These shifts reflect a broader aim of equipping educators with tools to measure student learning effectively and foster continuous learning beyond the confines of the classroom. However, despite the proliferation of methods integrating big data and AI technologies in educational settings, a discernible gap persists between technological capabilities and their practical implementation in education, notably in the realm of student assessment and evaluation. This gap highlights challenges in enhancing assessment accuracy and reliability, evaluating non-cognitive skills, and delivering personalized feedback. The introduction of AI-powered technologies like ChatGPT raises ethical considerations regarding fairness, privacy, and bias, necessitating responsible and equitable deployment.This paper addresses fundamental research questions concerning the historical influences on assessment frameworks, the role of AI-powered technology in driving paradigm shifts, and the ethical considerations surrounding its application. Through a comprehensive literature review, it explores the potential of AI technology to enhance assessment accuracy and reliability while advocating for its responsible use. Despite the challenges posed by traditional assessment processes and AI-powered technology assessment models, educators continually strive to improve student learning and readiness for lifelong learning.By celebrating the invaluable contributions of educators, we can shift the narrative surrounding assessment practices towards a more positive and supportive environment, fostering ethical and innovative uses of AI-powered technology. This celebration serves as inspiration for future research and development in assessment, emphasizing the importance of building upon educators’ innovative practices to shape the future of assessment in the big data era.

show abstract

The Role of Artificial Intelligence in Higher Education: ChatGPT Assessment for Anatomy Course

Cited by 52 publications

References 21 publications

ChatGPT failed Taiwan’s Family Medicine Board Exam

ChatGPT failed Taiwan’s Family Medicine Board Exam

Understanding learners’ perceptions of ChatGPT: A thematic analysis of peer interviews among undergraduates and postgraduates in China

The Paradigm Shift in Education in Big Data Era: Exploring the Intersection of Historical Assessment Frameworks and AI-Powered Assessment Methods in Education

Contact Info

Product

Resources

About