Development and Validation of a Test for Competence in Evidence-Based Medicine

Abstract

BACKGROUND: Medical educators need valid, reliable, and efficient tools to assess evidence-based medicine (EBM) knowledge and skills. Available EBM assessment tools either do not assess skills or are laborious to grade. OBJECTIVE: To validate a multiple-choice-based EBM test-the Resident EBM Skills Evaluation Tool (RESET). DESIGN: Cross-sectional study. PARTICIPANTS: A total of 304 medicine residents from five training programs and 33 EBM experts comprised the validation cohort. MAIN MEASURES: Internal reliability, item difficulty, and item discrimination were assessed. Construct validity was assessed by comparing mean total scores of trainees to experts. Experts were also asked to rate importance of each test item to assess content validity. KEY RESULTS: Experts had higher total scores than trainees (35.6 vs. 29.4, P < 0.001) and also scored significantly higher than residents on 11/18 items. Cronbach's alpha was 0.6 (acceptable), and no items had a low itemtotal correlation. Item difficulty ranged from 7 to 86%. All items were deemed "important" by > 50% of experts. CONCLUSIONS: The proposed EBM assessment tool is a reliable and valid instrument to assess competence in EBM. It is easy to administer and grade and could be used to guide and assess interventions in EBM education.

Prospective study on the organization and efficiency of online journal club

“…There is still no clear standard for assessing the knowledge of evidence-based medicine and statistics [2]. There are many methods proposed by different authors that still need to be tested on various cohorts [13,15,[20][21][22].…”

Assessing for Practice-Based Learning and Improvement: Distinguishing Evidence-Based Practice From Reflective Learning

“…6,7 The majority of the tools focus on multiple-choice and/or short-answer tests to evaluate the learner's knowledge of the tenets of evidence-based medicine. [8][9][10][11][12] The most studied of these tests is the UCSF-Fresno Medical Education tool, a 7-question written test evaluating how to ask a clinical question, assess the hierarchy of evidence, and understand basic statistical and methodological concepts. 9,10 The Fresno tool has been validated in several populations of learners and could be used to assess if a trainee has met Level 2 for the PBLI-1 Milestones.…”

Assessment Question Characteristics Predict Medical Student Performance in General Pathology

“…[7][8][9] Students use 3 main approaches to learning and studying: deep/thoughtful (seeking to understand, relating new concepts to prior knowledge, and critically examining evidence); superficial/surface (completing the task, memorizing information, and focusing on individual points without recognizing wider context or reflecting on the process); and strategic/efficient (organizing work, managing time, and aiming to efficiently pass any assessment). 10,11 In this context, student assessments that are perceived to reward the understanding of knowledge, rather than simple data regurgitation (ie, recall), would encourage deeper learning approaches. Furthermore, assessments that focus on and evaluate clinical reasoning, judgment, management of ambiguity, lifelong learning, and teamwork strategies would appraise learners in a more reliable and comprehensive manner.…”