Comparing Item Performance on Three- Versus Four-Option Multiple Choice Questions in a Veterinary Toxicology Course

Royal, Kenneth D.; Dorman, David C.

doi:10.3390/vetsci5020055

Cited by 3 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Royal & Dorman [18] highlighted that 3-option and 4-option MCQs had similar psychometric properties which means the former is equally effective as the latter. Therefore, the traditional MCQs with 4 options shall be maintained as more research needs to be done to better understand the effects of 3-option MCQs on guessing strategies and cut score determination decisions to avoid any unintended consequential validity [19].…”

Section: Resultsmentioning

confidence: 99%

Multiple-Choice Questions in Preclinical and Clinical Phase Examination for Undergraduate Medical Program in a Malaysian Medical School: A 2-Year Analysis

Adam

Idris

Kassim

et al. 2020

Preprint

View full text Add to dashboard Cite

Abstract Background: Multiple-choice questions (MCQs) are used in measuring the student’s progress and post-examination analysis is usually done to guarantee the item’s appropriateness for question banking. Item analysis is a simple and effective method to determine the quality of MCQs by using three parameters; difficulty or passing index (PI), discrimination index (DI) and distractor efficiency (DE). Methods: This study analysed the MCQs in the preclinical and clinical examinations in Doctor of Medicine program of Universiti Putra Malaysia. Forty MCQs consisted of four options each in the preclinical examination and 80 MCQs with five options from the clinical examination paper in 2017 and 2018 were analysed and compared. Results: The mean DI was similar in all examinations, except a significant reduction in 2018 clinical examination. From 2017 and 2018, preclinical MCQs showed an increment in the number of ‘excellent’ and ‘good’ items. However, clinical papers showed reduction in DI due to high number of ‘poor’ questions. Comparing both years, there was an increase in the number of items with no non-functioning distractors in both examinations. Among all, preclinical MCQs in 2018 showed the highest mean of DE as compared to the others.Conclusion: Our findings suggested that item authors from preclinical phase showed an improvement in constructing good quality MCQs while clinical phase authors may need more training and continuous feedback. Higher number of options did not affect the level of difficulty of a question however the discrimination power and effectiveness of distractors might differ.

show abstract

Section: Resultsmentioning

confidence: 99%

Multiple-Choice Questions in Preclinical and Clinical Phase Examination for Undergraduate Medical Program in a Malaysian Medical School: A 2-Year Analysis

Adam

Idris

Kassim

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Nevertheless, MCQs with two or more NFDs should be revised. A strategy to decrease the number of NFDs might be to reduce the number of answer options to three [29]. However, a concern for educators might be the increased odds that guessing has with 3-option versus 4-option MCQs (i.e.…”

Section: Discussionmentioning

confidence: 99%

Experiences from Two Ways of Integrating Pre- and Post-course Multiple-choice Assessment Questions in Educational Events for Surgeons

Ghidinelli

Cunningham

Monotti

et al. 2021

Journal of European CME

View full text Add to dashboard Cite

To examine how to optimise the integration of multiple-choice questions (MCQs) for learning in continuing professional development (CPD) events in surgery, we implemented and evaluated two methods in two subspecialities over multiple years. The same 12 MCQs were administered pre-and post-event in 66 facial trauma courses. Two different sets of 10 MCQs were administered pre-and post-event in 21 small animal fracture courses. We performed standard psychometric tests on responses from participants who completed both the pre-and post-event assessment. The average difficulty index pre-course was 57% with a discrimination index of 0.20 for small animal fractures and 53% with a discrimination index of 0.15 for facial trauma. For the majority of the individual MCQs, the scores were between 30%-70% and the discrimination index was >0.10. The difficulty index post-course increased in both groups (to 75% and 62%). The pre-course MCQs resulted in an average score in the expected range for both formats suggesting they were appropriate for the intended level of difficulty and an appropriate pre-course learning activity. Post-course completion resulted in increased scores with both formats. Both delivery methods worked well in all regions and overall quality depends on applying a solid item development and validation process.

show abstract

“…However, this means that students will have a high chance (only 1 in 3) of correctly answering the item with random guessing. Royal and Dorman highlighted that 3-option and 4-option MCQs had similar psychometric properties, which means the former is equally effective as the latter (20). Therefore, the traditional MCQs with four options shall be maintained as more research needs to be done to better understand the effects of 3option MCQs on guessing strategies and cut score determination decisions to avoid any unintended consequential validity (21).…”

Section: It Is Evident That Proper Vetting and Training In Constructingmentioning

confidence: 99%

Multiple Choice Questions with Different Numbers of Options in University Putra Malaysia Undergraduate Medical Program: A Comparative Analysis in 2017 and 2018

et al. 2021

View full text Add to dashboard Cite

Background: Multiple-choice questions (MCQs) are used for measuring the student’s progress, and they should be analyzed properly to guarantee the item’s appropriateness. The analysis usually determines three indices of an item; difficulty or passing index (PI), discrimination index (DI), and distractor efficiency (DE). Objectives: This study was aimed to analyze the multiple-choice questions in the preclinical and clinical examinations with different numbers of options in medical program of Universiti Putra Malaysia. Methods: This is a cross-sectional study. Forty multiple-choice questions with four options from the preclinical examination and 80 multiple-choice questions with five options from the clinical examination in 2017 and 2018 were analyzed using optical mark recognition machine and Ms. Excel. The parameters included PI, DI, and DE. Results: The average difficulty level of multiple-choice questions for preclinical and clinical phase examinations were similar in 2017 and 2018 that were considered ‘acceptable’ and ‘ideal’ ranged from 0.55 to 0.60, respectively. The average DIs were similar in all examinations that were considered ‘good’ (ranged from 0.25 to 0.31) except in 2018 clinical phase examination that showed ‘poor’ items (DI = 0.20 ± 0.11). The questions for preclinical phase showed an increase in the number of ‘excellent’ and ‘good’ items in 2018 from 37.5% to 70.0%. There was an increase of 10.0% for preclinical phase, and 6.25% for clinical phase, in the number of items with no non-functioning distractors in 2018. Among all, preclinical multiple-choice questions in 2018 showed the highest mean of DE (71.67%). Conclusions: Our findings suggested that there was an improvement in the questions from preclinical phase while more training on questions preparation and continuous feedback should be given to clinical phase teachers. A higher number of options did not affect the level of difficulty of a question; however, the discrimination power and distractors efficiency might differ.

show abstract

Comparing Item Performance on Three- Versus Four-Option Multiple Choice Questions in a Veterinary Toxicology Course

Cited by 3 publications

References 20 publications

Multiple-Choice Questions in Preclinical and Clinical Phase Examination for Undergraduate Medical Program in a Malaysian Medical School: A 2-Year Analysis

Multiple-Choice Questions in Preclinical and Clinical Phase Examination for Undergraduate Medical Program in a Malaysian Medical School: A 2-Year Analysis

Experiences from Two Ways of Integrating Pre- and Post-course Multiple-choice Assessment Questions in Educational Events for Surgeons

Multiple Choice Questions with Different Numbers of Options in University Putra Malaysia Undergraduate Medical Program: A Comparative Analysis in 2017 and 2018

Contact Info

Product

Resources

About