Network meta-analysis incorporating randomized controlled trials and non-randomized comparative cohort studies for assessing the safety and effectiveness of medical treatments: challenges and opportunities

Abstract

Network meta-analysis is increasingly used to allow comparison of multiple treatment alternatives simultaneously, some of which may not have been compared directly in primary research studies. The majority of network meta-analyses published to date have incorporated data from randomized controlled trials (RCTs) only; however, inclusion of non-randomized studies may sometimes be considered. Non-randomized studies can complement RCTs or address some of their limitations, such as short follow-up time, small sample size, highly selected population, high cost, and ethical restrictions. In this paper, we discuss the challenges and opportunities of incorporating both RCTs and non-randomized comparative cohort studies into network meta-analysis for assessing the safety and effectiveness of medical treatments. Non-randomized studies with inadequate control of biases such as confounding may threaten the validity of the entire network meta-analysis. Therefore, identification and inclusion of non-randomized studies must balance their strengths with their limitations. Inclusion of both RCTs and non-randomized studies in network meta-analysis will likely increase in the future due to the growing need to assess multiple treatments simultaneously, the availability of higher quality non-randomized data and more valid methods, and the increased use of progressive licensing and product listing agreements requiring collection of data over the life cycle of medical products. Inappropriate inclusion of non-randomized studies could perpetuate the biases that are unknown, unmeasured, or uncontrolled. However, thoughtful integration of randomized and non-randomized studies may offer opportunities to provide more timely, comprehensive, and generalizable evidence about the comparative safety and effectiveness of medical treatments.

The use of single armed observational data to closing the gap in otherwise disconnected evidence networks: a network meta-analysis in multiple myeloma

“…NMA models provide a powerful method to synthesize data from multiple trials and generate estimates of relative efficacy between treatments within connected networks of evidence, by combining direct and indirect evidence [34]. Indirect estimates rely on the assumption of transitivity and the use of relative effects ensures randomisation is preserved.…”

The “Golden Age” of Probiotics: A Systematic Review and Meta-Analysis of Randomized and Observational Studies in Preterm Infants

“…To address these investigative questions in preterm infants, especially in ELBW infants, randomized controlled trials (RCTs) are not always easy or ethical to conduct. Instead, well-designed observational studies may be the next practicable method to address these types of questions as they have been shown to provide results similar to RCTs, challenging the belief that observational studies are second-rate [11,12]. They reflect routine practice, which allows for the evaluation of effectiveness and safety in large populations that include ELBW patients who are under-represented in, or completely excluded from RCTs.…”

Immediate loading vs. early/conventional loading of immediately placed implants in partially edentulous patients from the patients’ perspective: A systematic review

“…For data derived from controlled trials, combining results of randomized and nonrandomized controlled trials has been questioned as it has been shown that results of nonrandomized controlled trials tended to show greater treatment effects than randomized controlled trials (Ioannidis et al., ). While newer Network Meta‐analysis may overcome this shortcoming (Cameron et al., ), two different sets of PROMs were used in the three comparative studies preventing pooling of the data and meaningful comparison between studies. Another shortcoming of these comparative trials was the fact that only one time point after treatment was considered for recording the PROMs which limited the prospective evaluation of the treatment benefits.…”