Choosing medical assessments: Does the multiple-choice question make the grade?

Pham, Hannah; Trigg, Monique; Wu, Shaopeng; O’Connell, Alice; Harry, Christopher; Barnard, J. A.; Devitt, Peter G.

doi:10.4103/efh.efh_229_17

Cited by 15 publications

(21 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…quality items testing higher order cognitive skills (12). However, when properly created, SAQs is a reliable, constructed-response assessment tool.…”

Section: Resultsmentioning

confidence: 99%

An Analysis of the Student’s Performance and Marking Process in Open-Ended Short Answer Questions in a Medical Programme: A Pilot Study

Win¹,

Yin²,

Venkateswaran³

et al. 2019

EIMJ

View full text Add to dashboard Cite

There has been no literature on analysing short answer questions (SAQs) to assess how well the students captured the essence of core concepts in basic sciences disciplines. The objective of this study was to assess the performance of students and marking process in core concept-intensive SAQs on physiology and pathology in a professional examination of the undergraduate medical curriculum. Two faculties, each from physiology and pathology disciplines identified the core concept-intensive SAQs. Consensus was taken on how core are the concepts and a marking scheme was developed by ranking of the elements of the concept. Based on their overall performance, top 20 students, middle 20 students and bottom 20 were identified, and their answers were graded (parameters: not attempted, wrong concept, incomplete or irrelevant, and how well the essence of critical key concepts was captured). This communication focused on the discrepancies in the marks awarded by the marker (examiner) and by the investigators. The discrepancy was higher in the middle 20 group while most of the bottom 20 had no marks awarded as they had not attempted or had a wrong or irrelevant concept. The construction of model answers in the examinations should be improved to unlock how much students could capture the underlying key critical concepts. Alternatively, the marker should be a content expert capable of going beyond the given model answer.

show abstract

“…quality items testing higher order cognitive skills (12). However, when properly created, SAQs is a reliable, constructed-response assessment tool.…”

Section: Resultsmentioning

confidence: 99%

An Analysis of the Student’s Performance and Marking Process in Open-Ended Short Answer Questions in a Medical Programme: A Pilot Study

Win¹,

Yin²,

Venkateswaran³

et al. 2019

EIMJ

View full text Add to dashboard Cite

show abstract

“…Pham et al, opined that drawbacks of an open-ended exam include less reliability and consumption of resources in construction, delivery, and marking which is reiterated in this study. 1 The low reliability of the question may be attributed to the fact that many students wrote action of the drug instead of writing the use of the drug.…”

Section: Will Award Full Marks If the Concept Is Correct Because Studmentioning

confidence: 99%

“…Pham et al, opined that the prerequisite of an SAQ to provide justification for each answer to obtain additional marks capitalizes on intermediate steps in thinking and hence if a single-format approach is used, the appropriateness and degree of "scaling" should be carefully considered. 1 Examiners act as a further source of error as they introduce internal adjustments which create inter-rater variability.…”

Section: Will Award Full Marks If the Concept Is Correct Because Studmentioning

confidence: 99%

“…Even though the Multiple Choice Questions has been well accepted in terms of reliability and resource effectiveness for cracking the entrance into the medical undergraduate and postgraduate courses, it has been perceived to be equivalent or inferior to other test instruments. 1,2 The Short Answer Questions (SAQ) is a focal tool that quantifies the cognitive skills and it capitalizes on spontaneous generation of answers bridging learning to real-life situations.…”

Section: Introductionmentioning

confidence: 99%

“…3 The versatility of SAQ lies in eliciting simple, short responses to demanding expression of a student's clinical reasoning and decision-making skills. 1 Though deemed to help facilitators in offering remediation, the inherent problems with SAQ lies in the resource intensiveness in construction and marking of written responses, subjectivity, and inter-observer variability in scoring candidates. 4 This study was done with an aim to understand the practices adopted by students in answering short answer question, to study the reliability and validity of SAQ under study and to obtain faculty feedback about valuing short answer question in Pharmacology.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Reliability and validity of short answer questions in Pharmacology

Sreedharan¹,

Palappallil²,

Munjely³

2019

Int J Basic Clin Pharmacol

View full text Add to dashboard Cite

Background: Short answer question (SAQ) is an excellent tool for assessment. Objective of this study was to understand the practices adopted in answering SAQ, assessment of reliability and validity of SAQ and faculty feedback on valuing SAQs.Methods: A cross-sectional study was done in the Department of Pharmacology of a Government Medical College in Central Kerala from December 2018-January 2019. The study participants were given 10 minutes to answer the two chosen SAQs after which the response sheets were collected and valued by 2 independent examiners. Feedback on correction of the same questions was sought from Pharmacology faculties of various Medical Colleges through online survey. The data were analysed using SPSS software.Results: 100% participants attempted to answer the SAQ ‘Definition of a drug’ but none of them came out with the World Health Organisation definition of drug. For the second SAQ, ‘Two uses of atropine’ around 25(20.8%) participants had written two uses of the drug correctly. The reliability was 0 for ‘definition of a drug’ and 0. 50 (95% CI 0.082-0.702) for ‘Two uses of atropine’ and both the questions had no validity. The feedback on valuing definition and two use of atropine was variable.Conclusions: None of the students wrote definition of a drug correctly and majority gave multiple responses to uses of atropine instead of two valid responses. The short answer questions we tested were found to have moderate to no reliability and no validity. The faculties had variable perceptions in valuing the two short answer questions.

show abstract

Educator's blueprint: A how‐to guide for developing high‐quality multiple‐choice questions

Gottlieb

Bailitz

Fix

et al. 2023

AEM Education and Training

View full text Add to dashboard Cite

Multiple-choice questions (MCQs) are a common modality used in undergraduate, graduate, and postgraduate medical education for assessing learners' knowledge across a number of applications.Educators create MCQs to assess learners' understanding of concepts and preparedness for higher-stakes examinations (e.g., United States Medical Licensing Examinations, in-training examinations, shelf examinations, board certification). Education researchers may use MCQs to assess knowledge before and after an intervention to gauge the intervention's effectiveness. Creators of online resources, including both paid (e.g., Physician's Evaluation and Educational Review, RoshReview) and open-access (e.g., https:// www.aliem.com/), may use MCQs as both tools for teaching (i.e., test-enhanced learning) and for learners to self-identify knowledge gaps to refine studying. Finally, learners may create their own tests to apply the learning or to share with others. Data have demonstrated the value of using MCQs for both knowledge assessment and test-enhanced learning. 1,2 Multiple-choice questions also offer the benefits of being time-efficient and easy to score and may allow more objective scoring compared with other open-ended testing methods. However, faculty rarely receive training in how to create a high-quality MCQ. This can lead to poorly constructed MCQs, which can hinder the ability to assess learning and guide study efforts by learners. 3,4 Moreover, poorly designed MCQs can also impact the ability to assess knowledge attained from an education intervention when used for program evaluation and research reporting. Therefore, there is a need to better understand the key components of creating effective MCQs for novice and experienced MCQ developers alike.

show abstract

Choosing medical assessments: Does the multiple-choice question make the grade?

Cited by 15 publications

References 0 publications

An Analysis of the Student’s Performance and Marking Process in Open-Ended Short Answer Questions in a Medical Programme: A Pilot Study

An Analysis of the Student’s Performance and Marking Process in Open-Ended Short Answer Questions in a Medical Programme: A Pilot Study

Reliability and validity of short answer questions in Pharmacology

Educator's blueprint: A how‐to guide for developing high‐quality multiple‐choice questions

Contact Info

Product

Resources

About