Dental Students’ Use of AMSTAR to Critically Appraise Systematic Reviews

Abstract

The idea of basing clinical procedures upon evidence gathered by observation is less than 200 years old, with the irst set of evidence-based position papers dating back only to the early 1970s. The relationship between evidence-based education and health outcomes is dificult to test and may be indirect, but teaching critical appraisal skills may be beneicial in developing knowledge. Systematic reviews have a central role in the process of clinical decision making in practice and therefore should be of high quality, following a rigorous protocol that can be evaluated with validated tools. The aim of this study was to assess how dental students utilized the Assessment of Multiple Systematic Reviews (AMSTAR) appraisal tool to evaluate systematic reviews in the context of a treatment planning course. During the in-class inal exam, students were required to appraise the quality of a systematic review and to justify their answers. Of the 74 third-year students who took the exam, 100% answered all questions on the AMSTAR form. The mean number of correct answers was nine (SD=1.047, Min=6, Max=10), with no student providing all 11 correct answers. The fact that nearly 90% of the students provided eight or more correct answers suggests that AMSTAR can be used by students to evaluate the methodological quality of systematic reviews. It also was evident that although the AMSTAR tool requires less than 15 minutes to complete an evaluation, using it requires extensive training and repetition to achieve consistent and reliable results.

AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both

“…In the later phases of development of AMSTAR 2 we had access to, and discussed, recently published critiques of AMSTAR. 25 26 27 28 29 30 31 …”

“… 23 The publications describing the original AMSTAR instrument were widely cited and the instrument has been used and critiqued extensively. 22 23 24 25 26 27 28 29 30 31 …”

“…Since publication, several critiques of the instrument have been published. 25 26 27 28 29 30 31 These critiques plus feedback received at workshops and developments in the science of systematic reviews pointed to a need to revise and update the original AMSTAR instrument.…”

Current Trends in Dental Education

“…Alternative methods for assessment of student performance are now being used as better and more innovative teaching methodologies are available. One such method was used to assess how third-year dental students at Case Western Reserve University utilize assessment of multiple systematic reviews (AMSTAR) to evaluate systematic reviews in the context of an evidence-based, dentistry-oriented, and treatment planning course 15 .…”

Using the Timmer Scale to Standardize Pediatric Dentistry Residents’ Scientific Appraisal Skills

“…Since the 1970s, multiple assessment instruments have been introduced to provide methodology for critical appraisal in systematic reviews of the professional literature, including: Consolidated Standards of Reporting Trials (CONSORT), 14 Timmer, 13 Jadad, 15 Assessment of Multiple Systematic Reviews (AMSTAR), 16 Revised Assessment of Multiple Systematic Reviews (R-AMSTAR), 17 Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRIMSA), 18 and Grading of Recommendations Assessment, Development and Evaluation (GRADE). 19 While most of these tools have multiple forms for different study designs like Oxford and the Journal of the American Medical Association , 21 – 25 Timmer, which has been validated and determined to be reliable for assessing research design methodology, has one form with standardized questions that critique the design, statistical methodology, and reporting of data. The specific study design is factored in using a specific formula that takes into account the randomization.…”