Searching practices and inclusion of unpublished studies in systematic reviews of diagnostic accuracy

Abstract

Many diagnostic accuracy studies are never reported in full in a peer-reviewed journal. Searching for unpublished studies may avoid bias due to selective publication, enrich the power of systematic reviews, and thereby help to reduce research waste. We assessed searching practices among recent systematic reviews of diagnostic accuracy.Methods: We extracted data from 100 non-Cochrane systematic reviews of diagnostic accuracy indexed in MEDLINE and published between October 2017 and January 2018 and from all 100 Cochrane systematic reviews of diagnostic accuracy published by December 2018, irrespective of whether metaanalysis had been performed.Results: Non-Cochrane and Cochrane reviews searched a median of 4 (IQR 3-5) and 6 (IQR 5-9) databases, respectively; most often MEDLINE/PubMed (n = 100 and n = 100) and EMBASE (n = 81 and n = 100). Additional efforts to identify studies beyond searching bibliographic databases were performed in 76 and 98 reviews, most often through screening reference lists (n = 71 and n = 96), review/guideline articles (n = 18 and n = 52), or citing articles (n = 3 and n = 42). Specific sources of unpublished studies were searched in 22 and 68 reviews, for example, conference proceedings (n = 4 and n = 18), databases only containing conference abstracts (n = 2 and n = 33), or trial registries (n = 12 and n = 39). At least one unpublished study was included in 17 and 23 reviews. Overall, 39 of 2082 studies (1.9%) included in non-Cochrane reviews were unpublished, and 64 of 2780 studies (2.3%) in Cochrane reviews, most often conference abstracts (97/103). Conclusion: Searching practices vary considerably across systematic reviews of diagnostic accuracy. Unpublished studies are a minimal fraction of the evidence included in recent reviews.

Omega-6 sparing effects of parenteral lipid emulsions—an updated systematic review and meta-analysis on clinical outcomes in critically ill patients

“…Published abstracts, which did not undergo peer-review, were considered for inclusion after careful assessment. This approach might potentially reduce publication bias, we do, however, acknowledge the ongoing controversy on this topic [ 71 – 73 ]. We report the largest and the most current aggregation of PN lipid trials conducted in the critical care setting.…”

A tale of two theories: A meta-analysis of the attention set and load theories of inattentional blindness.

“…included studies or relevant literature (e.g., reviews), or through our unpublished/gray literature search (see Table 1 and Figure 3). For unpublished and "gray" literature, following from previous recommendations (Korevaar et al, 2020), we employed three strategies: (a) contact with authors and discipline experts (N = 12, response rate = 50%), (b) search of online conference proceedings and reference guides (N = 4), and (c) search of university theses databases (N = 1). To ensure our search was exhaustive, we also conducted searches for preprint articles on the Open Science Framework and PsyArXiv using the term "inattentional blindness."…”

Validity of constructed-response situational judgment tests in training programs for the health professions: A systematic review and meta-analysis protocol

“…We will include unpublished studies as well as published papers to identify as much relevant evidence as possible. While grey literature may cause concerns regarding their methodological quality due to the absence of peer review, inclusion of unpublished data could reduce risk of publication bias, the “file-drawer problem,” enrich the power of findings, and reduce research waste [ 26 ]. To avoid any potential problem, we will follow the direction of the Cochrane handbook that recommends having at least a similar level of expertise in the review team as a peer reviewer for a journal to appraise unpublished studies [ 27 ].…”