Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review

Abstract

Background Stringent requirements exist regarding the transparency of the study selection process and the reliability of results. A 2-step selection process is generally recommended; this is conducted by 2 reviewers independently of each other (conventional double-screening). However, the approach is resource intensive, which can be a problem, as systematic reviews generally need to be completed within a defined period with a limited budget. The aim of the following methodological systematic review was to analyse the evidence available on whether single screening is equivalent to double screening in the screening process conducted in systematic reviews. Methods We searched Medline, PubMed and the Cochrane Methodology Register (last search 10/2018). We also used supplementary search techniques and sources (“similar articles” function in PubMed, conference abstracts and reference lists). We included all evaluations comparing single with double screening. Data were summarized in a structured, narrative way. Results The 4 evaluations included investigated a total of 23 single screenings (12 sets for screening involving 9 reviewers). The median proportion of missed studies was 5% (range 0 to 58%). The median proportion of missed studies was 3% for the 6 experienced reviewers (range: 0 to 21%) and 13% for the 3 reviewers with less experience (range: 0 to 58%). The impact of missing studies on the findings of meta-analyses had been reported in 2 evaluations for 7 single screenings including a total of 18,148 references. In 3 of these 7 single screenings – all conducted by the same reviewer (with less experience) – the findings would have changed substantially. The remaining 4 of these 7 screenings were conducted by experienced reviewers and the missing studies had no impact or a negligible on the findings of the meta-analyses. Conclusions Single screening of the titles and abstracts of studies retrieved in bibliographic searches is not equivalent to double screening, as substantially more studies are missed. However, in our opinion such an approach could still represent an appropriate methodological shortcut in rapid reviews, as long as it is conducted by an experienced reviewer. Further research on single screening is required, for instance, regarding factors influencing the number of studies missed. Electronic supplementary material The online version of this article (10.1186/s12874-019-0782-0) contains supplementary material, which is available to authorized users.

Combining abbreviated literature searches with single-reviewer screening: three case studies of rapid reviews

“…The findings of other methods studies assessing the validity of methodological shortcuts for literature searches [10,13,25,26] and single-reviewer screening [12][13][14][15] provide, in general, findings consistent with our study. For example, Pham et al [13] and Bayliss et al [25] identified 53% to 94%…”

“…Few studies have investigated the impact of single-reviewer literature screening [12][13][14][15]. A systematic review found that a median of 5% of the relevant studies were missed by single-reviewer screening (range 0% to 58%) [15]. None of the available studies, however, assessed the impact of "missed studies" on the results and conclusions of the evidence syntheses.…”

Prevalence and Nature of Medication Errors and Medication-Related Harm Following Discharge from Hospital to Community Settings: A Systematic Review

“…However, this study has a number of limitations that affected the internal validity including no independent quality assessment, and single author screening of citations, which could have led to the omission of relevant studies (though uncertain cases were discussed amongst the research team) [157]. A meta-analysis of outcome rate data was also not possible because of heterogeneity of included data.…”

Single-reviewer abstract screening missed 13 percent of relevant studies: a crowd-based, randomized controlled trial

“…To date, few methodological evaluations have determined the proportion of relevant studies that will be missed when titles and abstracts are screened by a sole investigator. A recent systematic review detected four studies of the accuracy of single-reviewer compared with dual-reviewer screening [12]. Across these publications, the median proportion of missed studies was 5%.…”