The cardiovascular outcomes challenge examined the predictive accuracy of 10 diabetes models in estimating hard outcomes in 2 recent cardiovascular outcomes trials (CVOTs) and whether recalibration can be used to improve replication. Methods: Participating groups were asked to reproduce the results of the Empagliflozin Cardiovascular Outcome Event Trial in Type 2 Diabetes Mellitus Patients (EMPA-REG OUTCOME) and the Canagliflozin Cardiovascular Assessment Study (CANVAS) Program. Calibration was performed and additional analyses assessed model ability to replicate absolute event rates, hazard ratios (HRs), and the generalizability of calibration across CVOTs within a drug class. Results: Ten groups submitted results. Models underestimated treatment effects (ie, HRs) using uncalibrated models for both trials. Calibration to the placebo arm of EMPA-REG OUTCOME greatly improved the prediction of event rates in the placebo, but less so in the active comparator arm. Calibrating to both arms of EMPA-REG OUTCOME individually enabled replication of the observed outcomes. Using EMPA-REG OUTCOME-calibrated models to predict CANVAS Program outcomes was an improvement over uncalibrated models but failed to capture treatment effects adequately. Applying canagliflozin HRs directly provided the best fit. Conclusions: The Ninth Mount Hood Diabetes Challenge demonstrated that commonly used risk equations were generally unable to capture recent CVOT treatment effects but that calibration of the risk equations can improve predictive accuracy. Although calibration serves as a practical approach to improve predictive accuracy for CVOT outcomes, it does not extrapolate generally to other settings, time horizons, and comparators. New methods and/or new risk equations for capturing these CV benefits are needed.
Background Structural uncertainty can affect model-based economic simulation estimates and study conclusions. Unfortunately, unlike parameter uncertainty, relatively little is known about its magnitude of impact on life-years (LYs) and quality-adjusted life-years (QALYs) in modeling of diabetes. We leveraged the Mount Hood Diabetes Challenge Network, a biennial conference attended by international diabetes modeling groups, to assess structural uncertainty in simulating QALYs in type 2 diabetes simulation models. Methods Eleven type 2 diabetes simulation modeling groups participated in the 9th Mount Hood Diabetes Challenge. Modeling groups simulated 5 diabetes-related intervention profiles using predefined baseline characteristics and a standard utility value set for diabetes-related complications. LYs and QALYs were reported. Simulations were repeated using lower and upper limits of the 95% confidence intervals of utility inputs. Changes in LYs and QALYs from tested interventions were compared across models. Additional analyses were conducted postchallenge to investigate drivers of cross-model differences. Results Substantial cross-model variability in incremental LYs and QALYs was observed, particularly for HbA1c and body mass index (BMI) intervention profiles. For a 0.5%-point permanent HbA1c reduction, LY gains ranged from 0.050 to 0.750. For a 1-unit permanent BMI reduction, incremental QALYs varied from a small decrease in QALYs (−0.024) to an increase of 0.203. Changes in utility values of health states had a much smaller impact (to the hundredth of a decimal place) on incremental QALYs. Microsimulation models were found to generate a mean of 3.41 more LYs than cohort simulation models ( P = 0.049). Conclusions Variations in utility values contribute to a lesser extent than uncertainty captured as structural uncertainty. These findings reinforce the importance of assessing structural uncertainty thoroughly because the choice of model (or models) can influence study results, which can serve as evidence for resource allocation decisions. Highlights The findings indicate substantial cross-model variability in QALY predictions for a standardized set of simulation scenarios and is considerably larger than within model variability to alternative health state utility values (e.g., lower and upper limits of the 95% confidence intervals of utility inputs). There is a need to understand and assess structural uncertainty, as the choice of model to inform resource allocation decisions can matter more than the choice of health state utility values.
NICE willingness to pay threshold range. For both models, results were very sensitive to changes in the small estimated incremental health benefit (0.0195 versus 0.0434 quality-adjusted life-years). Intrinsic differences between both modelling approaches contributed to differences in results. Data availability issues limited replication efforts. Model execution in the cohort-level model was near-instant. Conclusions: Were a cohort-based, non-DICE modelling approach submitted by the manufacturer in TA494, this research suggests the NICE recommendation would have likely been the same. However, the model execution issues faced by the ERG and affecting committee deliberations would have been avoided. Given the number of health states required to approximate the analysis in a cohort-level model, a non-DICE patientlevel approach directly capturing patient heterogeneity may have been more apt.
IntroductionDisease models can be useful tools for policy makers to inform their decisions. They can help to estimate the costs and benefits of interventions without conducting clinical trials and help to extrapolate the findings of clinical trials to a population level.Sexually transmitted infections (STIs) do not operate in isolation. Risk-taking behaviours and biological interactions can increase the likelihood of an individual being coinfected with more than one STI.Currently, few STI models consider coinfection or the interaction between STIs. We aim to identify and summarise STI models for two or more STIs and describe their modelling approaches.Methods and analysisSix databases (Cochrane, Embase, PLOS, ProQuest, Medline and Web of Science) were searched on 27 November 2018 to identify studies that focus on the reporting of the methodology and quality of models for at least two different STIs. The quality of all eligible studies will be accessed using a percentage scale published by Kopec et al. We will summarise all used approaches to model two or more STIs in one model. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework will be used to report all outcomes.Ethics and disseminationEthical approval is not required for this systematic review. The results of this review will be published in a peer-reviewed journal and presented at a suitable conference. The findings from this review will be used to inform the development of a new multi-STI model.PROSPERO registration numberCRD42017076837.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.