The overview of reviews: Unique challenges and opportunities when research syntheses are the principal elements of new integrative scholarship.

Abstract

In the past two decades, a new form of scholarship has appeared in which researchers present an overview of previously conducted research syntheses on the same topic. In these efforts, research syntheses are the principal units of evidence. Overviews of reviews introduce unique problems that require unique solutions. This article describes what methods overviewers have developed or have adopted from other forms of scholarship. These methods concern how to (a) define the broader problem space of an overview, (b) conduct literature searches that specifically look for research syntheses, (c) address the overlap in evidence in related reviews, (d) evaluate the quality of both primary research and research syntheses, (e) integrate the outcomes of research syntheses, especially when they produce discordant results, (f) conduct a second-order meta-analysis, and (g) present findings. The limitations of overviews are also discussed, especially with regard to the age of the included evidence.

WJNR Welcomes Umbrella Reviews

“…Also referred to in the literature as overviews of reviews, reviews of reviews, and systematic reviews of systematic reviews, umbrella reviews have the primary objective of addressing specific research questions by examination of the findings of previous reviews (Becker & Oxman, 2008). By synthesizing across previous reviews, patterns in findings can be identified that permit conclusions to be drawn about the state of research in an area, that help resolve discrepancies in conclusions drawn by different reviews, and can identify the direction future research should take (Becker & Oxman, 2008;Cooper & Koenka, 2012).…”

“…The validity and strength of conclusions drawn by any review article are dependent on the methodological quality of the review (Cooper & Koenka, 2012;Smith et al, 2011). Umbrella reviews should therefore always include a quality assessment of all included reviews.…”

“…Construction of multiple tables is often necessary to clearly present all the data collected from reviews (Cooper & Koenka, 2012;Smith et al, 2011). Well-constructed tables will facilitate analysis because they make patterns in the data easier to detect.…”

“…The research questions being addressed by the umbrella review will determine the inclusion criteria for reviews as well as inform the search strategy to be used to locate potentially eligible reviews. In locating reviews, sufficiently broad search strategies should be used to compensate for the limitations inherent to computerized databases, which may only provide information for a certain range of years or a particular set of journals (Cooper & Koenka, 2012;Smith et al, 2011). One search of a single database such as MEDLINE is rarely adequate; online searches should include multiple different databases.…”

“…Consistent application of inclusion criteria will yield an unbiased set of reviews for analysis (Smith et al, 2011). Umbrella review articles often include a flowchart of the inclusion of reviews and reasons why some reviews may have been excluded (Cooper & Koenka, 2012).…”