Search citation statements
Paper Sections
Citation Types
Year Published
Publication Types
Relationship
Authors
Journals
ImportanceRandomized clinical trials (RCTs) are widely regarded as the methodological benchmark for assessing clinical efficacy and safety of health interventions. There is growing interest in using nonrandomized studies to assess efficacy and safety of new drugs.ObjectiveTo determine how treatment effects for the same drug compare when evaluated in nonrandomized vs randomized studies.Data SourcesMeta-analyses published between 2009 and 2018 were identified in MEDLINE via PubMed and the Cochrane Database of Systematic Reviews. Data analysis was conducted from October 2019 to July 2024.Study SelectionMeta-analyses of pharmacological interventions were eligible for inclusion if both randomized and nonrandomized studies contributed to a single meta-analytic estimate.Data Extraction and SynthesisFor this meta-analysis using a meta-epidemiological framework, separate summary effect size estimates were calculated for nonrandomized and randomized studies within each meta-analysis using a random-effects model and then these estimates were compared. The reporting of this study followed the Guidelines for Reporting Meta-Epidemiological Methodology Research and relevant portions of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline.Main Outcome and MeasuresThe primary outcome was discrepancies in treatment effects obtained from nonrandomized and randomized studies, as measured by the proportion of meta-analyses where the 2 study types disagreed about the direction or magnitude of effect, disagreed beyond chance about the effect size estimate, and the summary ratio of odds ratios (ROR) obtained from nonrandomized vs randomized studies combined across all meta-analyses.ResultsA total of 346 meta-analyses with 2746 studies were included. Statistical conclusions about drug benefits and harms were different for 130 of 346 meta-analyses (37.6%) when focusing solely on either nonrandomized or randomized studies. Disagreements were beyond chance for 54 meta-analyses (15.6%). Across all meta-analyses, there was no strong evidence of consistent differences in treatment effects obtained from nonrandomized vs randomized studies (summary ROR, 0.95; 95% credible interval [CrI], 0.89-1.02). Compared with experimental nonrandomized studies, randomized studies produced on average a 19% smaller treatment effect (ROR, 0.81; 95% CrI, 0.68-0.97). There was increased heterogeneity in effect size estimates obtained from nonrandomized compared with randomized studies.Conclusions and RelevanceIn this meta-analysis of treatment effects of pharmacological interventions obtained from randomized and nonrandomized studies, there was no overall difference in effect size estimates between study types on average, but nonrandomized studies both overestimated and underestimated treatment effects observed in randomized studies and introduced additional uncertainty. These findings suggest that relying on nonrandomized studies as substitutes for RCTs may introduce additional uncertainty about the therapeutic effects of new drugs.
ImportanceRandomized clinical trials (RCTs) are widely regarded as the methodological benchmark for assessing clinical efficacy and safety of health interventions. There is growing interest in using nonrandomized studies to assess efficacy and safety of new drugs.ObjectiveTo determine how treatment effects for the same drug compare when evaluated in nonrandomized vs randomized studies.Data SourcesMeta-analyses published between 2009 and 2018 were identified in MEDLINE via PubMed and the Cochrane Database of Systematic Reviews. Data analysis was conducted from October 2019 to July 2024.Study SelectionMeta-analyses of pharmacological interventions were eligible for inclusion if both randomized and nonrandomized studies contributed to a single meta-analytic estimate.Data Extraction and SynthesisFor this meta-analysis using a meta-epidemiological framework, separate summary effect size estimates were calculated for nonrandomized and randomized studies within each meta-analysis using a random-effects model and then these estimates were compared. The reporting of this study followed the Guidelines for Reporting Meta-Epidemiological Methodology Research and relevant portions of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline.Main Outcome and MeasuresThe primary outcome was discrepancies in treatment effects obtained from nonrandomized and randomized studies, as measured by the proportion of meta-analyses where the 2 study types disagreed about the direction or magnitude of effect, disagreed beyond chance about the effect size estimate, and the summary ratio of odds ratios (ROR) obtained from nonrandomized vs randomized studies combined across all meta-analyses.ResultsA total of 346 meta-analyses with 2746 studies were included. Statistical conclusions about drug benefits and harms were different for 130 of 346 meta-analyses (37.6%) when focusing solely on either nonrandomized or randomized studies. Disagreements were beyond chance for 54 meta-analyses (15.6%). Across all meta-analyses, there was no strong evidence of consistent differences in treatment effects obtained from nonrandomized vs randomized studies (summary ROR, 0.95; 95% credible interval [CrI], 0.89-1.02). Compared with experimental nonrandomized studies, randomized studies produced on average a 19% smaller treatment effect (ROR, 0.81; 95% CrI, 0.68-0.97). There was increased heterogeneity in effect size estimates obtained from nonrandomized compared with randomized studies.Conclusions and RelevanceIn this meta-analysis of treatment effects of pharmacological interventions obtained from randomized and nonrandomized studies, there was no overall difference in effect size estimates between study types on average, but nonrandomized studies both overestimated and underestimated treatment effects observed in randomized studies and introduced additional uncertainty. These findings suggest that relying on nonrandomized studies as substitutes for RCTs may introduce additional uncertainty about the therapeutic effects of new drugs.
On January 30, 2024, over 300 researchers filed an amicus brief in FDA v. Alliance for Hippocratic Medicine, a United States (US) Supreme Court case that could have severely impacted access to mifepristone, one of the two drugs commonly used in medication abortion. The researchers summarize the legal challenges to the US Food and Drug Administration's (FDA's) original approval of mifepristone in 2000 and its 2016 and 2021 decisions modifying mifepristone's Risk Evaluation and Mitigation Strategy (REMS) Program and label, the responses from the FDA and drug manufacturer to the challenges, and the potential implications of the Court's decision on access to mifepristone in the US. The researchers detail how the FDA relied on a robust scientific record analyzing tens of thousands of patient experiences that conclusively demonstrated the safety and effectiveness of the changes to the mifepristone REMS Program and label and urge the Supreme Court to rely on the clear scientific record and preserve access to mifepristone without reimposing restrictions. What follows is a reprint of this brief.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.