Assessment of publication bias and outcome reporting bias in systematic reviews of health services and delivery research: A meta-epidemiological study

Abstract

Strategies to identify and mitigate publication bias and outcome reporting bias are frequently adopted in systematic reviews of clinical interventions but it is not clear how often these are applied in systematic reviews relating to quantitative health services and delivery research (HSDR). We examined whether these biases are mentioned and/or otherwise assessed in HSDR systematic reviews, and evaluated associating factors to inform future practice. We randomly selected 200 quantitative HSDR systematic reviews published in the English language from 2007-2017 from the Health Systems Evidence database (www.healthsystemsevidence.org). We extracted data on factors that may influence whether or not authors mention and/or assess publication bias or outcome reporting bias. We found that 43% (n = 85) of the reviews mentioned publication bias and 10% (n = 19) formally assessed it. Outcome reporting bias was mentioned and assessed in 17% (n = 34) of all the systematic reviews. Insufficient number of studies, heterogeneity and lack of pre-registered protocols were the most commonly reported impediments to assessing the biases. In multivariable logistic regression models, both mentioning and formal assessment of publication bias were associated with: inclusion of a meta-analysis; being a review of intervention rather than association studies; higher journal impact factor, and; reporting the use of systematic review guidelines. Assessment of outcome reporting bias was associated with: being an intervention review; authors reporting the use of Grading of Recommendations, Assessment, Development and Evaluations (GRADE), and; inclusion of only controlled trials. Publication bias and outcome reporting bias are infrequently assessed in HSDR systematic reviews. This may reflect the inherent heterogeneity of HSDR evidence and different methodological approaches to synthesising the evidence, lack of awareness of such biases, limits of current tools and lack of pre-registered study protocols for assessing such biases. Strategies to help raise awareness of the biases, and methods to

Delivering and implementing child and adolescent mental health training for mental health and allied professionals: a systematic review and qualitative meta-aggregation

“…Studies in a variety of formats were considered for inclusion, including peer-reviewed journal papers, doctoral theses, unpublished research, and conference papers, providing that extractable information was available. This approach mitigates the issue of publication bias, where the findings of peer-reviewed studies might differ substantially from those of unpublished research [33]. The years 2000 to present (January 2020) were used as date parameters in each search, minimising the chance of research findings being outdated.…”

Delivering and implementing child and adolescent mental health training for mental health and allied professionals: A systematic review and qualitative meta-aggregation

“…Studies in a variety of formats were considered for inclusion, including peer-reviewed journal papers, doctoral theses, unpublished research, and conference papers, providing that extractable information was available. This approach mitigates the issue of publication bias, where the ndings of peer-reviewed studies might differ substantially from those of unpublished research [33]. The years 2000 to present (January 2020) were used as date parameters in each search, minimising the chance of research ndings being outdated.…”

Assessment of Factors Causing Bias in Marketing- Related Publications

“…Publication or literature biases can be considered among the most frequently mentioned and discussed. Bias is also relevant in life sciences [15][16][17][18][19][20], psychology [21,22] education [23,24] and economics [25,26].…”