Comparison between Long-Menu and Open-Ended Questions in computerized medical assessments. A randomized controlled trial

Rotthoff, Thomas; Baehring, Thomas; Dicken, Hans-Dieter; Fahron, Urte; Richter, Bernd; Fischer, Martin R.; Scherbaum, Werner A.

doi:10.1186/1472-6920-6-50

Cited by 21 publications

(20 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…have already used KFs in a computer and case-based exam in a study involving students of human medicine using long selection lists (“long menus”) to investigate the feasibility, acceptance and test statistical quality and came to the conclusion that all of these three aspects are achieved and that therefore the KF approach with long selection lists is suitable for computer-based testing of applied knowledge. Rotthoff et al also report on the use of KFs with Long Menu Questions (LMQs) [18]. Fischer et al [19] conducted a study to develop and validate a KF-based exam for medical students.…”

Section: Discussionmentioning

confidence: 99%

Use of key feature questions in summative assessment of veterinary medicine students

2013

View full text Add to dashboard Cite

PurposeTo prove the hypothesis that procedural knowledge might be tested using Key Feature (KF) questions in written exams, the University of Veterinary Medicine Hannover Foundation (TiHo) pioneered this format in summative assessment of veterinary medicine students. Exams in veterinary medicine are either tested orally, practically, in written form or digitally in written form. The only question formats which were previously used in the written e-exams were Type A Single-choice Questions, Image Analysis and Short Answer Questions. E-exams are held at the TiHo using the electronic exam system Q [kju:] by CODIPLAN GmbH.MethodsIn order to examine less factual knowledge and more procedural knowledge and thus the decision-making skills of the students, a new question format was integrated into the exam regulations by the TiHo and some examiner used this for the first time in the computer based assessment. Following a successful pilot phase in formative e-exams for students, KF questions were also introduced in summative exams. A number of multiple choice questions were replaced by KF questions in four computer based assessment in veterinary medicine. The subjects were internal medicine, surgery, reproductive medicine and dairy science.ResultsThe integration and linking of KF questions into the computer based assessment system Q [kju:] went without any complications. The new question format was well received both by the students and the teaching staff who formulated the questions.ConclusionThe hypothesis could be proven that Key Feature questions represent a practicable addition to the existing e-exam question formats for testing procedural knowledge. The number of KF questions will be therefore further increased in examinations in veterinary medicine at the TiHo.

show abstract

Section: Discussionmentioning

confidence: 99%

Use of key feature questions in summative assessment of veterinary medicine students

2013

View full text Add to dashboard Cite

show abstract

“…They have the highest added value when in electronic exams where Long Menu answer prompts can be used as quasi open-ended questions (Schuwirth and van der Vleuten 2003;Rotthoff et al 2006). KF problems assessed with Short Menu or Multiple Choice formats are normally scored dichotomously.…”

Section: Scoring In Written Assessment Of Clinical Reasoningmentioning

confidence: 99%

Pick-N multiple choice-exams: a comparison of scoring algorithms

Bauer

Holzer

Kopp

et al. 2010

Adv in Health Sci Educ

View full text Add to dashboard Cite

To compare different scoring algorithms for Pick-N multiple correct answer multiple-choice (MC) exams regarding test reliability, student performance, total item discrimination and item difficulty. Data from six 3rd year medical students' end of term exams in internal medicine from 2005 to 2008 at Munich University were analysed (1,255 students, 180 Pick-N items in total). Scoring Algorithms: Each question scored a maximum of one point. We compared: (a) Dichotomous scoring (DS): One point if all true and no wrong answers were chosen. (b) Partial credit algorithm 1 (PS(50)): One point for 100% true answers; 0.5 points for 50% or more true answers; zero points for less than 50% true answers. No point deduction for wrong choices. (c) Partial credit algorithm 2 (PS(1/m)): A fraction of one point depending on the total number of true answers was given for each correct answer identified. No point deduction for wrong choices. Application of partial crediting resulted in psychometric results superior to dichotomous scoring (DS). Algorithms examined resulted in similar psychometric data with PS(50) only slightly exceeding PS(1/m) in higher coefficients of reliability. The Pick-N MC format and its scoring using the PS(50) and PS(1/m) algorithms are suited for undergraduate medical examinations. Partial knowledge should be awarded in Pick-N MC exams.

show abstract

“…LMQ [9] focuses on decision taking in the clinical process regarding diagnostics, diagnosis and therapy. Students type a word and a scroll down menu appears from which options can be chosen.…”

Section: Resultsmentioning

confidence: 99%

“…To answer the question, data from the case should be essential [8]. Questions constructed according to this principle are named Key Feature Questions (KFQ) in which a key feature is defined as a critical step in solving the problem [8, 9]. Adding material such as photos, videos and sounds to patient vignettes can make these vignettes more authentic.…”

Section: Introductionmentioning

confidence: 99%

Preferred question types for computer-based assessment of clinical reasoning: a literature study

Bruggen¹,

Woudenbergh²,

Spierenburg³

et al. 2012

Perspect Med Educ

View full text Add to dashboard Cite

Clinical reasoning is a core competence of doctors. Therefore, the assessment of clinical reasoning of undergraduate students is an important part of medical education. Three medical universities in the Netherlands wish to develop a shared question database in order to assess clinical reasoning of undergraduate students in Computer-Based Assessments (CBA). To determine suitable question types for this purpose a literature study was carried out. Search of ERIC and PubMed and subsequent cross referencing yielded 30 articles which met the inclusion criteria of a focus on question types suitable to assess clinical reasoning of medical students and providing recommendations for their use. Script Concordance Tests, Extended Matching Questions, Comprehensive Integrative Puzzles, Modified Essay Questions/Short Answer Questions, Long Menu Questions, Multiple Choice Questions, Multiple True/False Questions and Virtual Patients meet the above-mentioned criteria, but for different reasons not all types can be used easily in CBA. A combination of Comprehensive Integrative Puzzles and Extended Matching Questions seems to assess most aspects of clinical reasoning and these question types can be adapted for use in CBA. Regardless of the question type chosen, patient vignettes should be used as a standard stimulus format to assess clinical reasoning. Further research is necessary to ensure that the combination of these question types produces valid assessments and reliable test results.

show abstract

Comparison between Long-Menu and Open-Ended Questions in computerized medical assessments. A randomized controlled trial

Cited by 21 publications

References 12 publications

Use of key feature questions in summative assessment of veterinary medicine students

Use of key feature questions in summative assessment of veterinary medicine students

Pick-N multiple choice-exams: a comparison of scoring algorithms

Preferred question types for computer-based assessment of clinical reasoning: a literature study

Contact Info

Product

Resources

About