Evidence of selective reporting bias in hematology journals: A systematic review

Abstract

IntroductionSelective reporting bias occurs when chance or selective outcome reporting rather than the intervention contributes to group differences. The prevailing concern about selective reporting bias is the possibility of results being modified towards specific conclusions. In this study, we evaluate randomized controlled trials (RCTs) published in hematology journals, a group in which selective outcome reporting has not yet been explored.MethodsOur primary goal was to examine discrepancies between the reported primary and secondary outcomes in registered and published RCTs concerning hematological malignancies reported in hematology journals with a high impact factor. The secondary goals were to address whether outcome reporting discrepancies favored statistically significant outcomes, whether a pattern existed between the funding source and likelihood of outcome reporting bias, and whether temporal trends were present in outcome reporting bias. For trials with major outcome discrepancies, we contacted trialists to determine reasons for these discrepancies. Trials published between January 1, 2010 and December 31, 2015 in Blood; British Journal of Haematology; American Journal of Hematology; Leukemia; and Haematologica were included.ResultsOf 499 RCTs screened, 109 RCTs were included. Our analysis revealed 118 major discrepancies and 629 total discrepancies. Among the 118 discrepancies, 30 (25.4%) primary outcomes were demoted, 47 (39.8%) primary outcomes were omitted, and 30 (25.4%) primary outcomes were added. Three (2.5%) secondary outcomes were upgraded to a primary outcome. The timing of assessment for a primary outcome changed eight (6.8%) times. Thirty-one major discrepancies were published with a P-value and twenty-five (80.6%) favored statistical significance. A majority of authors whom we contacted cited a pre-planned subgroup analysis as a reason for outcome changes.ConclusionOur results suggest that outcome changes occur frequently in hematology trials. Because RCTs ultimately underpin clinical judgment and guide policy implementation, selective reporting could pose a threat to medical decision making.

Evaluation of selective outcome reporting and trial registration practices among addiction clinical trials

“…Wayant et al . found selective outcome reporting in 109 blood cancer trials with 118 major discrepancies. Howard et al .…”

“…compared outcomes listed in trial protocols with published reports . However, a large majority of studies have used the information included in clinical trial registries, such as ClinicalTrials.gov, to evaluate the discrepancies in outcomes between the trial registration and its publication . Use of the trial registration has two primary benefits for evaluation.…”

Blogging as a tool for the acquisition and dissemination of knowledge in health sciences: a preliminary evaluation

“…Despite the ethical consequences of not publishing negative results (Kaplan & Irvin, 2015;Mlinarić, Horvat, & Šupak Smolčić, 2017), although negative/null results are not negative for science, the existence of a publication bias toward the selective reporting of positive outcomes in the literature has previously been recognised and extensively discussed (Dwan, Gamble, Williamson, Kirkham,, & Reporting Bias Group, 2013;Hopewell, Loudon, Clarke, Oxman, & Dickersin, 2009;Wayant et al, 2017). A global analysis of the present experience, taking into account the increased workload for educators and the low participation of students in this type of activity, seems to support the idea that publication is biased toward successful experiences, which emphasises the attractive end product of the project (the blog itself ) and the positive evaluation of the students' experiences.…”

Ensuring the quality and specificity of preregistrations

“…For example, if a researcher does not state anything about the handling of outliers, it may imply they will not remove any outlying values or that they will consider what to do with outliers after finding them, which means that the RDF is not restricted. Furthermore, studies on registrations of clinical trials show that other RDFs like outcome switching and HARKing are common in publications of preregistered studies (Chan, 2008;Chan & Altman, 2005;Chan, Hrobjartsson, Haahr, Gotzsche, & Altman, 2004;Chan, Hróbjartsson, Jørgensen, Gøtzsche, & Altman, 2008;Goldacre, 2016;Ioannidis, Caplan, & Dal-Ré, 2017;Jones et al, 2017;Lancee, Lemmens, Kahn, Vinkers, & Luykx, 2017;Mayo-Wilson et al, 2017;Rankin et al, 2017;Wayant et al, 2017). It is possible that increasing the explicit commitment to exhaustiveness will reduce this tendency.…”