Modeling in Colorectal Cancer Screening: Assessing External and Predictive Validity of MISCAN-Colon Microsimulation Model Using NORCCAP Trial Results

Abstract

Background: Microsimulation models are increasingly being used to inform colorectal cancer (CRC) screening recommendations. MISCAN-Colon is an example of such a model, used to inform the Dutch CRC screening program and United States Preventive Services Task Force guidelines. Assessing the validity of these models is essential to provide transparency regarding their performance. In this study we tested the external and predictive validity of MISCAN-Colon. Methods: We validated MISCAN-Colon using the Norwegian Colorectal Cancer Prevention (NORCCAP) trial, a randomized controlled trial that examined the effectiveness of once-only flexible sigmoidoscopy (FS) screening. We simulated the study population and design of the NORCCAP trial in MISCAN-Colon and compared 10- to 12-year model predicted hazard ratios (HRs) for overall and distal CRC incidence and mortality to those observed. In addition, we compared the numbers of screen-detected neoplasia. Finally, we predicted the trial’s future results to allow for the assessment of predictive validity. Results: MISCAN-Colon predicted a HR for overall CRC incidence (0.85), for distal CRC incidence (0.82), for overall CRC mortality (0.68) and for distal CRC mortality (0.62). These were within the limits of the 95% confidence intervals of the NORCCAP trial results. Similar results were observed for the number of screen-detected cancers. The model significantly underestimated the number of screen-detected adenomas. Model-predicted HRs for CRC incidence and mortality up to 15- to 17-years follow-up were 0.84 and 0.72, respectively. Conclusion: Although the underestimation of screen-detected adenomas requires further investigation, MISCAN-Colon is able to make a valid replication of the CRC incidence and mortality reduction of an FS screening trial, which suggests that it can be considered a useful tool to support decision making on CRC screening.

Development and Validation of Three Regional Microsimulation Models for Predicting Colorectal Cancer Screening Benefits in Europe

“… 19 Specific model parameters, such as adenoma dwell time and the preclinical duration of CRC, were calibrated replicating outcomes of CRC screening RCTs 8 and, subsequently, validated to the results of the NORCCAP trial. 16 …”

“… a Calibrated together with colorectal cancer incidence in the prescreening period (please see Supplementary Tables 2 for more detailed information). b Parameters assumed equals to those previously calibrated in Rutter et al and validated in Buskermolen et al 8 , 16 c Stage distribution was adjusted considering a different staging system (No UICC TNM but Localized, Regional, Distant categorization). d Data on colorectal cancer mortality was not used inside the model (only for graphical inspection in Figure 1 ).…”

“…Those parameters were, however, inferred using results from randomized control trials (RTCs) investigating the effectiveness of CRC screening. 8 , 15 – 17 …”

Effects of Screening for Colorectal Cancer: Development and Validation of a Multistate Markov Model

“…However, the inherent downside of this reliance is that it complicates assessing the model's external validity, which evidently depends on the existence of data the model ought to reproduce. For instance, a microsimulation model calibrated to the Norwegian population was validated by replicating and predicting outcomes from the (Norwegian) NORCCAP RCT [37]. A major limitation of this study was that we could not provide a similar validation, as no comparably suitable large-scale trial at the highest level of evidence has been conducted in the German population.…”

Validation of Microsimulation Models against Alternative Model Predictions and Long-Term Colorectal Cancer Incidence and Mortality Outcomes of Randomized Controlled Trials

“…Previous colorectal cancer modeling studies have reported model validation against 1 or 2 RCT long-term follow-up findings. 25,45,46 A strength of this external validation exercise is that outcomes of 6 RCTs were included to validate model predictions, which significantly increases the value of the exercise and confidence in the outcome. Furthermore, the model predictions were compared against clinical trial findings with up to 30 years of follow-up.…”