Publication and related biases in health services research: a systematic review of empirical evidence

Abstract

Background: Publication and related biases (including publication bias, time-lag bias, outcome reporting bias and p-hacking) have been well documented in clinical research, but relatively little is known about their presence and extent in health services research (HSR). This paper aims to systematically review evidence concerning publication and related bias in quantitative HSR. Methods: Databases including MEDLINE, EMBASE, HMIC, CINAHL, Web of Science, Health Systems Evidence, Cochrane EPOC Review Group and several websites were searched to July 2018. Information was obtained from: (1) Methodological studies that set out to investigate publication and related biases in HSR; (2) Systematic reviews of HSR topics which examined such biases as part of the review process. Relevant information was extracted from included studies by one reviewer and checked by another. Studies were appraised according to commonly accepted scientific principles due to lack of suitable checklists. Data were synthesised narratively. Results: After screening 6155 citations, four methodological studies investigating publication bias in HSR and 184 systematic reviews of HSR topics (including three comparing published with unpublished evidence) were examined. Evidence suggestive of publication bias was reported in some of the methodological studies, but evidence presented was very weak, limited in both quality and scope. Reliable data on outcome reporting bias and p-hacking were scant. HSR systematic reviews in which published literature was compared with unpublished evidence found significant differences in the estimated intervention effects or association in some but not all cases. Conclusions: Methodological research on publication and related biases in HSR is sparse. Evidence from available literature suggests that such biases may exist in HSR but their scale and impact are difficult to estimate for various reasons discussed in this paper. Systematic review registration: PROSPERO 2016 CRD42016052333.

Rates of sustainment in the Universal Stages of Implementation Completion

“…In the case of the systematic reviews from which this paper is drawing comparisons, the reviewed studies might have been more likely to be published, and therefore included in the review, if the outcomes indicated successful sustainment. Health services research, which encompasses much of the sustainment literature that was reviewed, is thought to have strong incentives to publish results indicating beneficial outcomes, though the extent to which publication bias affects the field is unclear [ 37 ]. For this reason, such reviews are likely to overestimate the true rate of sustainment.…”

Introduction of the special issue “A Social Psychology of Leisure 2.0”

“…This innovative research design offers much potential for SPL research to reduce participant burden (i.e., survey length), particularly when examining the impact of leisure activity on socioemotional development and multiple variables/outcomes. While Dr. Gagnon's study did not find that parent or child level of camp experiences influenced parental perceptions of developmental outcomes, the study highlights the importance of publishing non-significant results (Amrhein et al, 2019;Ayorinde et al, 2020;Mlinari c et al, 2017) in order to disprove or revise existing theories and to provide a foundation for future studies to learn, replicate, revise and improve.…”

Paediatric tuberculosis diagnosis using Mycobacterium tuberculosis real-time polymerase chain reaction assay: a systematic review and meta-analysis

“…Our findings should be interpreted in the context of the quality of studies and reporting and variability in study quality. Diagnostic studies in general [72] and TB diagnostic studies in particular [73,74] seem to be beset by these problems.…”