A Rubric to Assess Critical Literature Evaluation Skills

Abstract

Objective. To develop and describe the use of a rubric for reinforcing critical literature evaluation skills and assessing journal article critiques presented by pharmacy students during journal club exercises. Design. A rubric was developed, tested, and revised as needed to guide students in presenting a published study critique during the second through fourth years of a first-professional doctor of pharmacy degree curriculum and to help faculty members assess student performance and provide formative feedback. Through each rubric iteration, the ease of use and clarity for both evaluators and students were determined with modifications made as indicated. Student feedback was obtained after using the rubric for journal article exercises, and interrater reliability of the rubric was determined. Assessment. Student feedback regarding rubric use for preparing a clinical study critique was positive across years. Intraclass correlation coefficients were high for each rubric section. The rubric was modified a total of 5 times based upon student feedback and faculty discussions. Conclusion. A properly designed and tested rubric can be a useful tool for evaluating student performance during a journal article presentation; however, a rubric can take considerable time to develop. A rubric can also be a valuable student learning aid for applying literature evaluation concepts to the critique of a published study.

Journal Clubs During Advanced Pharmacy Practice Experiences to Teach Literature-Evaluation Skills

“…The items measuring value were developed from both the AMSCOP journal club assessment criteria and the published literature. [13][14][15][16][17] Using a 4-point Likert scale (1 5 not helpful to 4 5 very helpful), other items assessed the following: format used for journal club (eg, type, length, documentation required); method of documentation for the journal club assignment (eg, PowerPoint slides, outline, list of responses to questions); which APPE was completed first; student perceptions of whether the drug-information APPE would have prepared them better for journal club; and value of the classroom statistics course completed the year before APPEs. Confidence in ability to conduct journal club in the future was measured using a 5-point Likert scale (1 5 strongly disagree to 5 5 strongly agree).…”

The Development of a Competency-Based Assessment Rubric to Measure Resident Milestones

“…Within the health professions, rubrics have been used for assessing literature review skills, 9 grading papers, 20 assessing presentations, 21,22 improving the quality of online courses, 23 participating in online discussions, 24 determining clinical performance in the operating room, 16 and measuring skill development of aseptic techniques. 25 Overall, rubrics promote consistency in scoring, encourage selfimprovement and self-assessment, motivate learners to achieve the next level, provide timely feedback, and improve instruction.…”

“…Identifiable issues related to resident assessments include inadequate description of evaluation criteria, variations in raters' observations and assessments, unsatisfactory or lack of meaningful feedback, and timeliness of feedback. [7][8][9] Furthermore, assessment tools appear to lack detailed requirements of performance expectations as well as behaviors for each competency or domain. 1 There continues to be an overemphasis on evaluating knowledge acquisition rather than measuring performance progress over time.…”

Medical Literature Evaluation Education at US Schools of Pharmacy

“…Innovations in teaching MLE include incorporating active-learning strategies into lecture-based courses, [13][14][15] providing course lecture material online rather than in the classroom, 16 using rubrics to aid students in assessment of a given piece of literature, 17,18 and requiring journal club participation during the advanced pharmacy practice experience (APPE) year. 19 Innovations in assessment of student ability to master literature evaluation include repeated testing embedded within and throughout the curriculum, 20 and assessment partway through the APPE year to measure long-term retention of previously-taught material.…”