A Comparison Between Written Assessment Methods: Multiple-choice and Short Answer Questions in End-of-clerkship Examinations for Final Year Medical Students

Abstract

Introduction An important aspect of a modern academic curriculum is assessment, which can be clinical and written. Written assessment includes both multiple-choice questions (MCQs) and short answer questions (SAQs). Debate continues as to which is more reliable. It is important to assess the correlation between the two different formats of written assessments, especially in the clinical subjects as they are different from the basic science subjects. Moreover, data are lacking in the correlation of the two formats of the written assessment in the clinical subjects. Therefore, we conducted this study to see the correlation between MCQs and SAQs in the end-of-clerkship examinations for final-year medical students. Materials and methods The end-of-clerkship written assessment results of the four disciplines of medicine, surgery, gynecology, and pediatrics were included. This was a retrospective correlational analytical study conducted at Shifa Tameer-e-Millat University, Islamabad, from 2013 to 2017. Data were analyzed using IBM SPSS Statistics for Windows, version 23.0 (IBM Corp., Armonk, NY); mean, standard deviation, Pearson coefficient, and p values were calculated both for MCQs and SAQs. Results A total of 481 students were involved in our study. The mean percentage scores of MCQs and SAQs in medicine were the most similar, and scores in obstetrics and gynecology had the most disparity. As compared to MCQs, the wider standard deviations were found in SAQs. Pearson correlations were 0.49, 0.47, 0.23, and 0.38 for medicine, surgery, gynecology, and pediatrics, respectively. Conclusion While we found mild to moderate significant correlation between MCQs and SAQs for final-year medical students, further investigations are required to explore the correlation and enhance the validity of our written assessments.

Discriminant Validity and Reliability of Scores of Multiple Choice and Short Essay Questions

“…5 Study by Mahmood (2015) shows moderate correlation between the MCQs and SEQS, however inter-rater reliability of SEQs showed high correlation. 8 Similar study was done by Farooqui et al 9 to assess the correlation of MCQs and SEQs in medicine, surgery, pediatrics and obstetrics and gynecology examinations. Moderate positive significant correlation was found between MCQs and SEQs in three subjects except for obstetrics and gynecology where weak positive correlation was found.…”

“…Author suggested that additional studies are needed to determine the correlation between these two modes of examination and increase the validity of assessments via SEQ's. 9 Moderate positive correlation (r = 0.5 P < 0.01) between scores of MCQs and SEQs was found in the send up undergraduate examination for Ophthalmology. 8 On the other hand astudy showed significant positive correlation of performance between SEQs with MCQs in the subject of Pharmacology at Kasturba Medical College at Manipal University, Karnataka, India.…”

An Analysis of the Student’s Performance and Marking Process in Open-Ended Short Answer Questions in a Medical Programme: A Pilot Study

“…A study showed that more students had scores greater than the median in the SAQs scores (14). Another study found that a statistically significant overall correlation between students' performance on MCQs and SAQs in major clinical subjects (9).…”

“…In SAQs marking, there is a wider standard deviation by bias or variable marking by different examiners whereas marking is more consistent in MCQs (9). However, SAQs have high validity as students need to be able to generate the piece of knowledge in the absence of cues, an approach that is more representative of real-life medical practice (5).…”

“…There is an extensive literature on postexamination analysis (psychometrics) for MCQs (7) as well as essay questions (8); but not on SAQs to assess how well students captured the essence of key concepts in basic sciences disciplines. Some studies on SAQs reported that mean standard deviation of SAQs score is wider than that of MCQs (9). Hence, this study was carried out to fill this gap as it is becoming evident that with knowledge explosion, learning should be by understanding and application of the core or threshold concepts in a discipline and not by content memorisation (10)(11).…”