Validation of an Evidence-Based Medicine Critically Appraised Topic Presentation Evaluation Tool (EBM C-PET)

Abstract

Background Competence in evidence-based medicine (EBM) is an important clinical skill. Pediatrics residents are expected to acquire competence in EBM during their education, yet few validated tools exist to assess residents' EBM skills. Objective We sought to develop a reliable tool to evaluate residents' EBM skills in the critical appraisal of a research article, the development of a written EBM critically appraised topic (CAT) synopsis, and a presentation of the findings to colleagues. Methods Instrument development used a modified Delphi technique. We defined the skills to be assessed while reviewing (1) a written CAT synopsis and (2) a resident's EBM presentation. We defined skill levels for each item using the Dreyfus and Dreyfus model of skill development and created behavioral anchors using a frame-of-reference training technique to describe performance for each skill level. We evaluated the assessment instrument's psychometric properties, including internal consistency and interrater reliability. Results The EBM Critically Appraised Topic Presentation Evaluation Tool (EBM C-PET) is composed of 14 items that assess residents' EBM and global presentation skills. Resident presentations (N = 27) and the corresponding written CAT synopses were evaluated using the EBM C-PET. The EBM C-PET had excellent internal consistency (Cronbach α = 0.94). Intraclass correlation coefficients were used to assess interrater reliability. Intraclass correlation coefficients for individual items ranged from 0.31 to 0.74; the average intraclass correlation coefficients for the 14 items was 0.67. Conclusions We identified essential components of an assessment tool for an EBM CAT synopsis and presentation with excellent internal consistency and a good level of interrater reliability across 3 different institutions. The EBM C-PET is a reliable tool to document resident competence in higher-level EBM skills.

A study on how using an interactive multimedia e-book improves teachers’ ability to teach evidence-based medicine depending on their seniority

“…The internal consistency of all indexes was estimated using Cronbach's α [23][24][25]. The questionnaire had a Cronbach's coefficient α of 0.801.…”

A Study on How Using an Interactive Multimedia E-Book Improves Teachers’ Ability to Teach Evidence-Based Medicine, Depending on Their Seniority

“…The nal survey included only items with strong relevance. The internal consistency of all indexes was estimated using Cronbach's α [22][23][24].…”

Attitudes and Perception of Evidence Based Dentistry in a Group of Egyptian Orthodontists: A Pilot Survey

“…(8)(9)(10)(11) Frequently identified barriers include; time constraints, Lack of access to resources, lack of evidence-based practice skills, difficulty and lack of confidence in interpreting the evidence, lack of knowledge, lack of hard evidence, resistance and criticism from colleagues, resistance to change and lack of trust in evidence or research and contradictory information in the scientific literature. (8)(9)(10)(11)(12)(13) The adoption of evidence based health practice by leading international medical and dental schools (14)(15)(16) and prominent organizations, (17,18) promises that evidence-based dentistry (EBD) will soon become an international requirement of practice and assessment. In the Egyptian orthodontic community, defining the learning needs, developing dental school curricula and continuing education programs are essential to improve awareness, attitudes and the skills needed to promote this practice in the future.…”