Extent of publication bias in different categories of research cohorts: a meta-analysis of empirical studies

Abstract

BackgroundThe validity of research synthesis is threatened if published studies comprise a biased selection of all studies that have been conducted. We conducted a meta-analysis to ascertain the strength and consistency of the association between study results and formal publication.MethodsThe Cochrane Methodology Register Database, MEDLINE and other electronic bibliographic databases were searched (to May 2009) to identify empirical studies that tracked a cohort of studies and reported the odds of formal publication by study results. Reference lists of retrieved articles were also examined for relevant studies. Odds ratios were used to measure the association between formal publication and significant or positive results. Included studies were separated into subgroups according to starting time of follow-up, and results from individual cohort studies within the subgroups were quantitatively pooled.ResultsWe identified 12 cohort studies that followed up research from inception, four that included trials submitted to a regulatory authority, 28 that assessed the fate of studies presented as conference abstracts, and four cohort studies that followed manuscripts submitted to journals. The pooled odds ratio of publication of studies with positive results, compared to those without positive results (publication bias) was 2.78 (95% CI: 2.10 to 3.69) in cohorts that followed from inception, 5.00 (95% CI: 2.01 to 12.45) in trials submitted to regulatory authority, 1.70 (95% CI: 1.44 to 2.02) in abstract cohorts, and 1.06 (95% CI: 0.80 to 1.39) in cohorts of manuscripts.ConclusionDissemination of research findings is likely to be a biased process. Publication bias appears to occur early, mainly before the presentation of findings at conferences or submission of manuscripts to journals.

Thirty percent of abstracts presented at dental conferences are published in full: a systematic review

“…[25] The fact that, in the relevant included studies [16,19], the number of positive abstracts (452) were 8.5 times that of negative abstracts (53) might be an indirect evidence.…”

“…In addition, although many studies with long follow-up supported a follow-up length of at least four to five years for publication fate research, [2,15,22,24] previous systematic reviews regarding publication fate had no strict eligibility criterion in terms of follow-up length (minimum follow-up of included studies: eight months to two years). [13,14,25] Also no former research has studied the influence of methods used in a publication fate study on the FPR obtained. [13] This study uses multivariable meta-regression to analyze study-level factors (conference characteristics, methodological quality), and sets an inclusion criterion of having at least 48-month follow-up from the date of conference.…”

Guest Editor’s Introduction to Special Issue: The Science and Practice of Research Synthesis

“… publication status is not a good proxy for study quality (McLeod & Weisz, 2004); and  the odds of publication are 2 to 3 times greater for studies that have positive, statistically significant results (Song, Parekh, et al, 2010;Song, Parekh-Bhurke, et al, 2009).…”

Deep brain stimulation compared with bariatric surgery for the treatment of morbid obesity: a decision analysis study

“…In reality, the true efficacy requirements of DBS may be < 83% to achieve equivalence. Furthermore, the well-established tendency of the medical literature to favor the report of positive studies, known as publication bias, 101 may be considered an alternative explanation. Based on sensitivity analysis, LAGB did not result in a superior outcome at any value when compared with the other 2 procedures.…”