Agreement of treatment effects from observational studies and randomized controlled trials evaluating hydroxychloroquine, lopinavir-ritonavir, or dexamethasone for covid-19: meta-epidemiological study

Abstract

ObjectiveTo systematically identify, match, and compare treatment effects and study demographics from individual or meta-analysed observational studies and randomized controlled trials (RCTs) evaluating the same covid-19 treatments, comparators, and outcomes.DesignMeta-epidemiological study.Data sourcesNational Institutes of Health Covid-19 Treatment Guidelines, a living review and network meta-analysis published in The BMJ, a living systematic review with meta-analysis and trial sequential analysis in PLOS Medicine (The LIVING Project), and the Epistemonikos “Living OVerview of Evidence” (L·OVE) evidence database.Eligibility criteria for selection of studiesRCTs in The BMJ’s living review that directly compared any of the three most frequently studied therapeutic interventions for covid-19 across all data sources (that is, hydroxychloroquine, lopinavir-ritonavir, or dexamethasone) for any safety and efficacy outcomes. Observational studies that evaluated the same interventions, comparisons, and outcomes that were reported in The BMJ’s living review.Data extraction and synthesisSafety and efficacy outcomes from observational studies were identified and treatment effects for dichotomous (odds ratios) or continuous (mean differences or ratios of means) outcomes were calculated and, when possible, meta-analyzed to match the treatment effects from individual RCTs or meta-analyses of RCTs reported in The BMJ’s living review with the same interventions, comparisons, and outcomes (that is, matched pairs). The analysis compared the distribution of study demographics and the agreement between treatment effects from matched pairs. Matched pairs were in agreement if both observational and RCT treatment effects were significantly increasing or decreasing (P<0.05) or if both treatment effects were not significant (P≥0.05).Results17 new, independent meta-analyses of observational studies were conducted that compared hydroxychloroquine, lopinavir-ritonavir, or dexamethasone with an active or placebo comparator for any safety or efficacy outcomes in covid-19 treatment. These studies were matched and compared with 17 meta-analyses of RCTs reported in The BMJ’s living review. 10 additional matched pairs with only one observational study and/or one RCT were identified. Across all 27 matched pairs, 22 had adequate reporting of demographical and clinical data for all individual studies. All 22 matched pairs had studies with overlapping distributions of sex, age, and disease severity. Overall, 21 (78%) of the 27 matched pairs had treatment effects that were in agreement. Among the 17 matched pairs consisting of meta-analyses of observational studies and meta-analyses of RCTs, 14 (82%) were in agreement; seven (70%) of the 10 matched pairs consisting of at least one observational study or one RCT were in agreement. The 18 matched pairs with treatment effects for dichotomous outcomes had a higher proportion of agreement (n=16, 89%) than did the nine matched pairs with treatment effects for continuous outcomes (n=5, 56%).ConclusionsMeta-analyses of observational studies and RCTs evaluating treatments for covid-19 have summary treatment effects that are generally in agreement. Although our evaluation is limited to three covid-19 treatments, these findings suggest that meta-analyzed evidence from observational studies might complement, but should not replace, evidence collected from RCTs.

Anorexia Nervosa With Comorbid Severe Depression

“…The second important issue is that causality cannot be inferred from these studies, the majority of which were observational. One possible suggestion for research going forward—aside from RCT initiatives—could be controlled studies with clearly predefined rules for stopping for benefit and other safety protocols and overseen by independent data review committees 27,57 . More well-designed and preregistered observational studies using accurate outcome measures could also be beneficial 58 .…”

“…One possible suggestion for research going forward-aside from RCT initiativescould be controlled studies with clearly predefined rules for stopping for benefit and other safety protocols and overseen by independent data review committees. 27,57 More well-designed and preregistered observational studies using accurate outcome measures could also be beneficial. 58 From this perspective, psychometric research focused on measurement of comorbid MDD in AN could be of utility for the standardization of measurement methods in research and clinical practice, allowing for increased comparability across samples.…”

Observational studies of traditional Chinese medicine may provide evidence nearly consistent with the randomized controlled trials: A meta-epidemiological study

“…Osman et al . conducted a meta-epidemiological study that evaluated the agreement between treatment effects of hydroxychloroquine, lopinavir-ritonavir and dexamethasone for COVID-19 from observational studies and that of RCTs 29 . They didn't pool ROR for different kinds of intervention, however, in the 11 matched pairs, the median ROR was 0.96 (RCTs as the reference group).…”

“…Several studies were published to assess the agreement of evidence between RCTs and observational studies in the field of nutrition, healthcare, and COVID-19 26 , 27 , 28 , 29 . However, whether the evidence from observational studies agreed with RCTs in the field of TCM was still unclear.…”

Replacing RCTs with real world data for regulatory decision making: a self-fulfilling prophecy?

“…Studies on whether observational real world data are suitable for investigating treatment effects use different approaches. One approach is to compare the treatment effects derived from RCTs with those from observational datasets, which has produced conflicting results 1718. A limitation of this approach is that any difference in effects between the two study types might reflect differences in patient characteristics or the application of interventions rather than tell us anything about the validity of observational studies.…”