Objective: To evaluate therapeutic meta-analyses including both observational studies and randomized controlled trials (RCTs), how these studies were combined and whether there were differences in treatment effects. Study design and setting: Meta-epidemiological study of meta-analyses including both observational studies and RCTs. We searched MEDLINE for the 5 leading journals of each medical category according to Journal Citation Reports) and Cochrane Database of Systematic Reviews, from 2014 to 2018 for eligible meta-analyses and extracted how observational studies and RCTs were combined and results for each study. Results: Of the 102 included meta-analyses, observational studies and RCTs were combined together without a subgroup analysis in 39 (38%) and with subgroup analysis in 15 (15%); they were pooled separately for the same outcome in 11 (11%) and not for the same outcome in 9 (9%). In 28 (27%) meta-analyses, only RCTs were combined, with qualitative description of observational studies. Treatment effect estimates did not differ between observational studies and RCTs (ratio of estimates=0.98 [95% confidence interval 0.80-1.21]), with substantial heterogeneity (I 2 =59%). Conclusion: Many meta-analyses including both observational studies and RCTs pool results from both study types. Although treatment effects did not differ between them on average, we identified situations for which estimates differed.
BackgroundPublication bias and other reporting bias have been well documented for journal articles, but no study has evaluated the nature of results posted at ClinicalTrials.gov. We aimed to assess how many randomized controlled trials (RCTs) with results posted at ClinicalTrials.gov report statistically significant results and whether the proportion of trials with significant results differs when no treatment effect estimate or p-value is posted.MethodsWe searched ClinicalTrials.gov in June 2015 for all studies with results posted. We included completed RCTs with a superiority hypothesis and considered results for the first primary outcome with results posted. For each trial, we assessed whether a treatment effect estimate and/or p-value was reported at ClinicalTrials.gov and if yes, whether results were statistically significant. If no treatment effect estimate or p-value was reported, we calculated the treatment effect and corresponding p-value using results per arm posted at ClinicalTrials.gov when sufficient data were reported.ResultsFrom the 17,536 studies with results posted at ClinicalTrials.gov, we identified 2823 completed phase 3 or 4 randomized trials with a superiority hypothesis. Of these, 1400 (50%) reported a treatment effect estimate and/or p-value. Results were statistically significant for 844 trials (60%), with a median p-value of 0.01 (Q1-Q3: 0.001–0.26). For the 1423 trials with no treatment effect estimate or p-value posted, we could calculate the treatment effect and corresponding p-value using results reported per arm for 929 (65%). For 494 trials (35%), p-values could not be calculated mainly because of insufficient reporting, censored data, or repeated measurements over time. For the 929 trials we could calculate p-values, we found statistically significant results for 342 (37%), with a median p-value of 0.19 (Q1-Q3: 0.005–0.59).ConclusionsHalf of the trials with results posted at ClinicalTrials.gov reported a treatment effect estimate and/or p-value, with significant results for 60% of these. p-values could be calculated from results reported per arm at ClinicalTrials.gov for only 65% of the other trials. The proportion of significant results was much lower for these trials, which suggests a selective posting of treatment effect estimates and/or p-values when results are statistically significant.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.