Calibration of Disease Simulation Model Using an Engineering Approach

Abstract

Objectives Calibrating a disease simulation model’s outputs to existing clinical data is vital to generate confidence in the model’s predictive ability. Calibration involves two challenges: 1) defining a total goodness-of-fit score for multiple targets if simultaneous fitting is required; and 2) searching for the optimal parameter set that minimizes the total goodness-of-fit score (i.e., yields the best fit). To address these two prominent challenges, we have applied an engineering approach to calibrate a microsimulation model, the Lung Cancer Policy Model (LCPM). Methods First, eleven targets derived from clinical and epidemiological data were combined into a total goodness-of-fit score by a weighted-sum approach, accounting for the user-defined relative importance of the calibration targets. Second, two automated parameter search algorithms, Simulated Annealing (SA) and Genetic Algorithm (GA), were independently applied to a simultaneous search of 28 natural history parameters to minimize the total goodness-of-fit score. Algorithm performance metrics were defined for speed and model fit. Results Both search algorithms obtained total goodness-of-fit scores below 95 within 1,000 search iterations. Our results show that SA outperformed GA in locating a lower goodness-of-fit. After calibrating our LCPM, the predicted natural history of lung cancer was consistent with other mathematical models of lung cancer development. Conclusion An engineering-based calibration method was able to simultaneously fit LCPM output to multiple calibration targets, with the benefits of fast computational speed and reduced need for human input and its potential bias.

Personalizing annual lung cancer screening for patients with chronic obstructive pulmonary disease: A decision analysis

“…Model outputs are calibrated to observed outcomes and tumor registry data from the Surveillance, Epidemiology, and End Results (SEER) program of the National Cancer Institute. 18 Additional details of the LCPM and model inputs have been previously described 19-20 and are available online 14 and summarized in the Appendix (see eAppendix, eTable 1, eFigure 1). In the sections below, we describe the approach used to incorporate spirometry-defined COPD as a risk factor for LC.…”

Using Active Learning for Speeding up Calibration in Simulation Models

“…Simulation models have been increasingly used in health policy decision making [1, 2, 3]. Many of these models include a component that describes the natural history of the disease and simulates an individual’s course of health to assess the overall effect of the disease in the population [4, 5, 6, 7].…”

Using Radiation Risk Models in Cancer Screening Simulations: Important Assumptions and Effects on Outcome Projections

“…Each individual may develop up to three cancers from any of five lung cancer cell types (adenocarcinoma, large cell, squamous cell, small cell, or other cancers). Natural history parameters related to unobservable events (ie, the initiation of the first cancer cell) were estimated through model calibration by using tumor registry data and data from published cohort studies (23). The model has been validated, and a detailed description is available online (https://cisnet.flexkb.net/mp/ pub/cisnet_lung_mghita_profile.pdf).…”