Validation of Models Used to Inform Colorectal Cancer Screening Guidelines

Abstract

Background Microsimulation models synthesize evidence about disease processes and interventions, providing a method for predicting long-term benefits and harms of prevention, screening, and treatment strategies. Because models often require assumptions about unobservable processes, assessing a model’s predictive accuracy is important. Methods We validated three colorectal cancer (CRC) microsimulation models against outcomes from the United Kingdom Flexible Sigmoidoscopy Screening (UKFSS) Trial, a randomized controlled trial that examined the effectiveness of one-time flexible sigmoidoscopy screening to reduce CRC mortality. The models incorporate different assumptions about the time from adenoma initiation to development of preclinical and symptomatic CRC. Analyses compare model predictions to study estimates across a range of outcomes to provide insight into the accuracy of model assumptions. Results All three models accurately predicted the relative reduction in CRC mortality ten years after screening (predicted hazard ratios, with 95% percentile intervals: 0.56 (0.44,0.71), 0.63 (0.51,0.75), 0.68 (0.53,0.83); estimated with 95% confidence interval: 0.56 (0.45,0.69). Two models with longer average preclinical duration accurately predicted the relative reduction in ten-year CRC incidence. Two models with longer mean sojourn time accurately predicted the number of screen-detected cancers. All three models predicted too many proximal adenomas among patients referred to colonoscopy. Conclusion Model accuracy can only be established through external validation analyses such as these and are therefore essential for any decision model. Results supported the assumptions that the average time from adenoma initiation to development of preclinical cancer is long (up to 25 years), and mean sojourn time is close to 4 years, suggesting the window for early detection and intervention by screening is relatively long. Variation in dwell time remains uncertain and could have important clinical and policy implications.

Early detection versus primary prevention in the PLCO flexible sigmoidoscopy screening trial: Which has the greatest impact on mortality?

“…However, only around 30% of intervention arm cases were screen detected, and some (11%) usual care arm cases were also screen detected. Further, lead time is estimated to be relatively short (1.6–4.0 years) for CRC and overdiagnosis is not believed to be substantial 9 . Therefore, the quantitative effects of lead-time and overdiagnosis were likely modest.…”

Estimation of Benefits, Burden, and Harms of Colorectal Cancer Screening Strategies

“…The 2008 analysis for the USPSTF 9 used SimCRC and MISCAN, although MISCAN has since been revised 10 based on findings from a joint model validation study. 11 …”

“…The models have been validated 11 against the findings from the UK Flexible Sigmoidoscopy Screening (UKFSS) Trial of once-only SIG. 5 All 3 models predicted CRC mortality reductions 10 years after SIG screening that were within the trial’s 95% confidence interval.…”

Impact of adenoma detection on the benefit of faecal testing vs. colonoscopy for colorectal cancer

“…13 Results have also been validated against the estimated mortality reduction from the UK flexible sigmoidoscopy screening trial (Supplementary Figure 3). 14,15 Although no experimental data were available to directly validate the modelled effectiveness of FIT and colonoscopy screening, effect sizes were concordant with observational data (Supplementary Figure 4–5). 16,17 Predicted mortality risks after polypectomy were validated against long-term National Polyp Study observations (Supplementary Figure 6).…”