Abstract:Objective:
Systematic reviews and meta-analyses (SRMAs) rely upon comprehensive searches into diverse resources that catalog primary studies. However, since what constitutes a comprehensive search is unclear, we examined trends in databases searched from 2005–2016, surrounding the publication of search guidelines in 2013, and associations between resources searched and evidence of publication bias in SRMAs involving human subjects.
Study Design:
To ensure comparability of included SRMAs over the 12 years in … Show more
“…Grey literature and unpublished data help to avoid publication bias, because searching sources that only cover published results may just return more of the same evidence. By contrast, grey literature searches and searches of clinical trial registries may reduce bias by retrieving evidence from a more diverse range of sources (Pradhan et al, 2018 ). This could influence the conclusions and consequently health care decisions (Halfpenny et al, 2016 ).…”
Background: Information specialists conducting searches for systematic reviews need to consider key questions around which and how many sources to search. This is particularly important for public health topics where evidence may be found in diverse sources.
Objectives:The objective of this review is to give an overview of recent studies on information retrieval guidance and methods that could be applied to public health evidence and used to guide future searches.
Methods: A literature search was performed in core databases and supplemented by browsing health information journals and citation searching. Results were sifted and reviewed. Results: Seventy-two papers were found and grouped into themes covering sources and search techniques. Public health topics were poorly covered in this literature. Discussion: Many researchers follow the recommendations to search multiple databases. The review topic influences decisions about sources. Additional sources covering grey literature eliminate bias but are time-consuming and difficult to search systematically. Public health searching is complex, often requiring searches in multidisciplinary sources and using additional methods.Conclusions: Search planning is advisable to enable decisions about which and how many sources to search. This could improve with more work on modelling search scenarios, particularly in public health topics, to examine where publications were found and guide future research.
“…Grey literature and unpublished data help to avoid publication bias, because searching sources that only cover published results may just return more of the same evidence. By contrast, grey literature searches and searches of clinical trial registries may reduce bias by retrieving evidence from a more diverse range of sources (Pradhan et al, 2018 ). This could influence the conclusions and consequently health care decisions (Halfpenny et al, 2016 ).…”
Background: Information specialists conducting searches for systematic reviews need to consider key questions around which and how many sources to search. This is particularly important for public health topics where evidence may be found in diverse sources.
Objectives:The objective of this review is to give an overview of recent studies on information retrieval guidance and methods that could be applied to public health evidence and used to guide future searches.
Methods: A literature search was performed in core databases and supplemented by browsing health information journals and citation searching. Results were sifted and reviewed. Results: Seventy-two papers were found and grouped into themes covering sources and search techniques. Public health topics were poorly covered in this literature. Discussion: Many researchers follow the recommendations to search multiple databases. The review topic influences decisions about sources. Additional sources covering grey literature eliminate bias but are time-consuming and difficult to search systematically. Public health searching is complex, often requiring searches in multidisciplinary sources and using additional methods.Conclusions: Search planning is advisable to enable decisions about which and how many sources to search. This could improve with more work on modelling search scenarios, particularly in public health topics, to examine where publications were found and guide future research.
“…Unfortunately, despite the significance of PB, previous studies show that a considerable proportion of systematic reviewers in different fields, such as oncology 7 , anesthesiology 8 , dermatology 9 , cardiology 10 , and gastroenterology 11 did not try to evaluate its possible presence in their SRs and MAs. Also, a substantial proportion of SRs were found not to search resources other than published materials, hence increasing the risk of PB in their results 12 .…”
Introduction: One concern in meta-analyses is the presence of publication bias (PB) which leads to the dissemination of inflated results. In this study, we assessed how much the meta-analyses in the field of otorhinolaryngology in 2021 evaluated the presence of PB. Methods: Six of the most influential journals in the field were selected. A search was conducted, and data were extracted from the included studies. In cases where PB was not assessed by the authors, we evaluated the risk of its presence by designing funnel plots and performing statistical tests. Results: 75 systematic reviews were included. 51% of them used at least one method for assessing the risk of PB, with the visual inspection of a funnel plot being the most frequent method used. 29% of the studies reported a high risk of PB presence. We replicated the results of 11 meta-analyses that did not assess the risk of PB and found that 63.6% were at high risk. Discussion: Our results indicate that systematic reviews published in some of the most influential journals in the field do not implement enough measures in their search strategies to reduce the risk of PB, nor do they assess the risk of its presence.
“…Unfortunately, despite the significance of PB, previous studies show that a considerable proportion of systematic reviewers in different healthcare fields, such as oncology [ 7 ], anesthesiology [ 8 ], dermatology [ 9 ], cardiology [ 10 ], and gastroenterology [ 11 ], did not try to evaluate its possible presence in their SRs and MAs. Also, a substantial proportion of SRs were found not to search resources other than published materials, hence increasing the risk of PB in their results [ 12 ].…”
Introduction
One concern in meta-analyses is the presence of publication bias (PB) which leads to the dissemination of inflated results. In this study, we assessed how much the meta-analyses in the field of otorhinolaryngology in 2021 evaluated the presence of PB.
Methods
Six of the most influential journals in the field were selected. A search was conducted, and data were extracted from the included studies. In cases where PB was not assessed by the authors, we evaluated the risk of its presence by designing funnel plots and performing statistical tests.
Results
Seventy-five systematic reviews were included. Fifty-one percent of them used at least one method for assessing the risk of PB, with the visual inspection of a funnel plot being the most frequent method used. Twenty-nine percent of the studies reported a high risk of PB presence. We replicated the results of 11 meta-analyses that did not assess the risk of PB and found that 63.6% were at high risk. We also found that a considerable proportion of the systematic reviews that found a high risk of PB did not take it into consideration when making conclusions and discussing their results.
Discussion
Our results indicate that systematic reviews published in some of the most influential journals in the field do not implement enough measures in their search strategies to reduce the risk of PB, nor do they assess the risk of its presence or take the risk of its presence into consideration when inferring their results.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.