Critical Appraisal in Clinical Practice: Sometimes Irrelevant, Occasionally Invalid

Abstract

SUMMARYA core activity of evidence-based practice is the search for and appraisal of evidence on speci®c clinical issues.Clinicians vary in their competence in this process; we therefore developed a 16-item checklist for quality of content (relevance and validity) and presentation (useability, attribution, currency and contact details). This was applied to a set of 55 consecutive appraisals conducted by clinicians and posted at a web-based medical journal club site.Questions were well formulated in 51/55 (92%) of the appraisals. However, 22% of appraisals missed the most relevant articles to answer the clinical question. Validity of articles was well appraised, with methodological information and data accurately extracted in 84% and accurate conversion to clinically meaningful summary statistics in 87%. The appraisals were presented in a useable way with appropriate and clear bottom-lines stated in 95%.The weakest link in production of good-quality critical appraisals was identi®cation of relevant articles. This should be a focus for evidence-based medicine and critical appraisal skills.

Journal Clubs During Advanced Pharmacy Practice Experiences to Teach Literature-Evaluation Skills

“…The items measuring value were developed from both the AMSCOP journal club assessment criteria and the published literature. [13][14][15][16][17] Using a 4-point Likert scale (1 5 not helpful to 4 5 very helpful), other items assessed the following: format used for journal club (eg, type, length, documentation required); method of documentation for the journal club assignment (eg, PowerPoint slides, outline, list of responses to questions); which APPE was completed first; student perceptions of whether the drug-information APPE would have prepared them better for journal club; and value of the classroom statistics course completed the year before APPEs. Confidence in ability to conduct journal club in the future was measured using a 5-point Likert scale (1 5 strongly disagree to 5 5 strongly agree).…”

The journal club and its practices

“…24 In an evidence based journal club the club as a part of evidence based practice was started in Birmingham Women's Hospital in July 1998. 10 …”

“…There may be a 4 cycle format, where the final day is for the presentation of the appraisal, peer review, refinement and to make a single page summary called 'Critically Appraised Topic (CAT)'. 10,13 A reverse format is used in some hospital based journal club where the clinical question comes first then the suitable study design and other essential things are discussed to build up a framework to critically appraise the chosen articles. 13,15 In another model question formulation and search strategy is guided by the speaker's presentation on a clinical scenario.…”

A Rubric to Assess Critical Literature Evaluation Skills

“…To prepare this list, several sources were located that detailed questions or issues to take into account when evaluating a published study. [8][9][10][11] Specific questions were also added based upon areas that were consistently overlooked or inappropriately discussed during the journal club presentations. Version 4 of the rubric was used by the 2 primary drug information preceptors to evaluate the fourth-professional year student journal club presentations during the drug information rotation.…”

“…Several articles have listed important questions and criteria to use when evaluating published clinical studies. [8][9][10][11] However, using such questions or criteria in the form of a simple checklist (ie, indicating present or absent) does not provide judgments of the quality or depth of coverage of each item. 12 A rubric is a scoring tool that contains criteria for performance with descriptions of the levels of performance that can be used for performance assessments.…”