Reporting of statistically significant results at ClinicalTrials.gov for completed superiority randomized controlled trials

Abstract

BackgroundPublication bias and other reporting bias have been well documented for journal articles, but no study has evaluated the nature of results posted at ClinicalTrials.gov. We aimed to assess how many randomized controlled trials (RCTs) with results posted at ClinicalTrials.gov report statistically significant results and whether the proportion of trials with significant results differs when no treatment effect estimate or p-value is posted.MethodsWe searched ClinicalTrials.gov in June 2015 for all studies with results posted. We included completed RCTs with a superiority hypothesis and considered results for the first primary outcome with results posted. For each trial, we assessed whether a treatment effect estimate and/or p-value was reported at ClinicalTrials.gov and if yes, whether results were statistically significant. If no treatment effect estimate or p-value was reported, we calculated the treatment effect and corresponding p-value using results per arm posted at ClinicalTrials.gov when sufficient data were reported.ResultsFrom the 17,536 studies with results posted at ClinicalTrials.gov, we identified 2823 completed phase 3 or 4 randomized trials with a superiority hypothesis. Of these, 1400 (50%) reported a treatment effect estimate and/or p-value. Results were statistically significant for 844 trials (60%), with a median p-value of 0.01 (Q1-Q3: 0.001–0.26). For the 1423 trials with no treatment effect estimate or p-value posted, we could calculate the treatment effect and corresponding p-value using results reported per arm for 929 (65%). For 494 trials (35%), p-values could not be calculated mainly because of insufficient reporting, censored data, or repeated measurements over time. For the 929 trials we could calculate p-values, we found statistically significant results for 342 (37%), with a median p-value of 0.19 (Q1-Q3: 0.005–0.59).ConclusionsHalf of the trials with results posted at ClinicalTrials.gov reported a treatment effect estimate and/or p-value, with significant results for 60% of these. p-values could be calculated from results reported per arm at ClinicalTrials.gov for only 65% of the other trials. The proportion of significant results was much lower for these trials, which suggests a selective posting of treatment effect estimates and/or p-values when results are statistically significant.

Global mapping of randomised trials related articles published in high-impact-factor medical journals: a cross-sectional analysis

“…differences in journal or author characteristics between publicly funded and industry-funded studies, designs/methodology, etc. ), the reporting and the reproducible research practices through research of research ("meta-research") studies [92][93][94][95][96][97][98].…”

Trends in mental health clinical research: Characterizing the ClinicalTrials.gov registry from 2007–2018

“…Since then, ClinicalTrials.gov has grown to become one of the largest international registries for clinical research, and currently it contains detailed information on more than 335,000 clinical studies conducted in over 200 countries. It has been analyzed in over 300 research articles to characterize the landscape of clinical research [7,8], and these articles have shed light on publication bias [9], noncompliance with trials registration [10], and selective reporting [11]. They have also identified trends across medical research [12] and within individual fields [12][13][14] of how trials are funded and designed, as well as what medical disorders and interventions are studied.…”

Global mapping of randomised trials related articles published in high-impact factor medical journals: a cross-sectional analysis

“…There is a potential length time-effect bias which puts more recent articles at a disadvantage. In addition, the biomedical literature is rich in barriers and motivations for publication and citation preferences [87], including self-citation (bias towards one's own work) [88], language bias (bias towards publishing and citing English articles), omission bias (bias purposely not citing competitors), and selective reporting and publication bias (bias withholding "negative" results from publication and citation) [89][90][91][92]. In addition, citations are also treated as equal regardless of whether a research is being cited for its positive contribution to the field, but also for being criticized.…”

“…In addition, citations are also treated as equal regardless of whether a research is being cited for its positive contribution to the field, but also for being criticized. Finally, our methods represent only a mapping approach which could be complemented further by more detailed analyses, for example examining the content, the reporting and reproducible research practices through research of research ("meta-research") studies [92][93][94][95].…”