Over the past several decades, metrics have been defined to assess the quality of various types of models and to compare their performance depending on their capacity to explain the variance found in real-life data. However, available validation methods are mostly designed for statistical regressions rather than for mechanistic models. To our knowledge, in the latter case, there are no consensus standards, for instance for the validation of predictions against real-world data given the variability and uncertainty of the data. In this work, we focus on the prediction of time-to-event curves using as an application example a mechanistic model of non-small cell lung cancer. We designed four empirical methods to assess both model performance and reliability of predictions: two methods based on bootstrapped versions of parametric statistical tests: log-rank and combined weighted log-ranks (MaxCombo); and two methods based on bootstrapped prediction intervals, referred to here as raw coverage and the juncture metric. We also introduced the notion of observation time uncertainty to take into consideration the real life delay between the moment when an event happens, and the moment when it is observed and reported. We highlight the advantages and disadvantages of these methods according to their application context. With this work, we stress the importance of making judicious choices for a metric with the objective of validating a given model and its predictions within a specific context of use. We also show how the reliability of the results depends both on the metric and on the statistical comparisons, and that the conditions of application and the type of available information need to be taken into account to choose the best validation strategy.
Lung adenocarcinoma (LUAD) is associated with a low survival rate at advanced stages. Although the development of targeted therapies has improved the survival of LUAD patients with identified and specific genetic alterations, such as mutations on the epidermal growth factor receptor (EGFR), the emergence of tumor resistance remains a threat to patient survival and calls for the development of new therapies. In this paper, we present the In Silico EGFR mutant LUAD (ISELA) model that links patients’ individual characteristics, including tumor genetic heterogeneity, to tumor size evolution and tumor progression over time under first generation EGFR-tyrosine kinase inhibitor gefitinib. This translational mechanistic model gathers extensive knowledge on LUAD and was calibrated on multiple scales, including in vitro, human tumor xenograft mouse and human, reproducing more than 90% of the experimental data identified. Moreover, with 98.5% coverage and 99.4% negative log rank tests, the model accurately reproduced the time to progression from the Lux-Lung 7 clinical trial, which was unused in calibration, thus supporting the model high predictive value. Therefore, this knowledge-based mechanistic model could be a valuable tool to support the development of new therapies against EGFR mutant LUAD as a foundation for the generation of synthetic control arms based on mechanistic models.
e20565 Background: Tumor heterogeneity and treatment resistance remains a continued threat to patients with advanced non-small cell lung cancer (NSCLC), despite the emergence of targeted medicines, e.g. against epidermal growth factor receptor (EGFR) alterations. We developed an in silico EGFR mutated lung adenocarcinoma (EGFR+ LUAD) model to predict the effect of known oncogenic EGFR mutations (common EGFR mutations, EGFR exon 20 insertions). The model provides mechanistic representation of tumor progression, including response to gefitinib and captures tumor heterogeneity, patient age, gender, initial clinical disease stage, and smoking status. Methods: 5-step in silico model development: Model building: biology of EGFR+LUAD was characterized by extracting biological features and their functional relationships from > 300 published papers and translating them into ordinary differential equations (ODEs). Mutational burden, EGFR-downstream-pathways, tumor growth and heterogeneity, gefitinib-PK/PD, treatment-induced resistance and clinical outcome were incorporated into a knowledge-based model containing 27-97 variables, 108-258 parameters and 13-83 ODEs reflecting intra-tumor clonal heterogeneity in a mechanistic manner. Calibration: published spheroid, xenograft and clinical data were used for stepwise calibration to find the correct parameter values. Relevant virtual populations (VPOPs) matching real patients baseline characteristics were generated for model benchmarking and validation. Benchmarking against a published data-based model: coverage of experimental interquartile range (IQR) with simulated IQR (precision) assesses model fit with experimental data, coverage of simulated IQR with experimental IQR (overlap) assesses model fit with experimental variability. Validation: a VPOP with comparable baseline characteristics was simulated and results compared against a published patient dataset that wasn’t used in any step during calibration. Matching of prediction over clinical data was done using both coverage and bootstrapped log-rank metrics. Results: Our model provides comparable outputs to the data-based model without having access to the exact original data (our model: precision of 62%, overlap of 91% VS data-based model: precision of 72% and an overlap of 86%). Besides, as a validation criterion, our model also successfully reproduces the time to progression observed in an independent clinical trial (coverage > 99%, negative log-rank tests > 98%). Conclusions: We simulated tumor growth and treatment response in advanced EGFR+ LUAD patients and successfully validated results both against an existing data-based model and a published clinical data. Our model highlights the potential of in silico modeling to better understand complex diseases progression and support efficient development of innovative therapies.
Introduction: Non-small cell lung cancer (NSCLC) response to gefitinib depends on epidermal growth factor receptor (EGFR) mutational status. Response differs according to tumor heterogeneity, notably through clonal selection according to genetic background. We developed an in silico EGFR mutated lung adenocarcinoma (EGFR+ LUAD) model topredict the effect of EGFR mutations on tumor size in advanced LUAD patients using a mechanistic representation of tumor progression, including response to gefitinib. Tumor heterogeneity, age, gender, initial clinical stage, and smoking status are included as covariates. Methods: 5-step in silico model development: (i) Model Building: Biology of EGFR+LUAD was characterized by extracting biological features and their functional relationships from literature and translating them into ordinary differential equations (ODEs). Mutational burden, EGFR downstream-pathways, tumor growth and heterogeneity, gefitinib-PK/PD, treatment-induced resistance and clinical outcome were modeled in a computational simulation with 43 variables, 170 parameters and 18 to 83 ODEs reflecting intra-tumor heterogeneity. (ii) Calibration: Published spheroid, xenograft and clinical data were used for stepwise calibration. (iii) Virtual populations (VPOP): VPOPs were generated for validation and benchmarking respectively, adapting baseline characteristics of a real population.(iv) Validation: A VPOP with comparable baseline characteristics was tested against published patient data.(v) Benchmarking against a Bayesian reference model : (1) coverage of experimental interquartile range (IQR) with simulated IQR (precision) assesses model fit with experimental data, (2) coverage of simulated IQR with experimental IQR (overlap) assesses model fit with experimental variability. Results: Our model computed in silico data comparable to the reference model without use of original data for calibration (Figure 1B.2: experimental vs. simulated, precision of 68%, overlap of 91%). The reference model reported precision of 72% and overlap of 86%. Therefore, the ISELA model has a better percentage of the literature data area contained in the simulated one while the Bayesian model presents a better percentage of the simulated data contained in the literature one. Conclusion: We simulated tumor growth and treatment response in advanced EGFR+ LUAD patients and successfully validated results with published data, and compared it to an already published model with the same context of use. Both models successfully provide a reliable description of longitudinal tumor size when compared to each other or to observed data. Our model provides a benchmark for future in silico clinical trials. Citation Format: Michael Duruisseaux, Adèle L'Hostis, Emmanuel Peyronnet, Evgueni Jacob, Ben M.W. Illigens, Jim Bosley, Riad Kahoul, Jean-Louis Palgen, Claudio Monteiro. Multiscale EGFR mutated NSCLC tumor heterogeneity knowledge-based model predicts tumor growth under gefitinib: An avenue to in silico clinical trial [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 3362.
Introduction Epidermal growth factor receptor (EGFR) mutations occur in about 40% of Asian and 13-25% of Western patients with lung adenocarcinoma (LUAD) [1,2]. EGFR tyrosine kinase inhibitors (TKIs) have been developed to target tumors with an EGFR driver mutation. However such tumors also harbor additional mutations and genotypic alterations, which contribute to the variability in treatment response. Overall, intratumor heterogeneity is a dynamic source of therapeutic resistance. Here, we assessed the impact of tumor mutational profile on inter-patient treatment response variability by using a mechanistic model of late stage EGFR-mutant LUAD [3]. Methods We developed in the jinko platform a knowledge-based model of late stage EGFR-mutant LUAD, whose parameters each hold a pathophysiology-related meaning. Indeed, causality between disease-related biological phenomena is inherent to the mechanistic knowledge-based model, easing the biological interpretation of the impact of parameter values on model outcomes, which is highly valuable in the context of uncommon populations. To explore the impact of tumor heterogeneity, tumors are implemented with distinct subpopulations of cancer cells, called subclones. Each subclone is defined by a set of mutations and a corresponding proliferating phenotype. A virtual population representative of real world EGFR-mutant LUAD patients was developed to reproduce the variability observed between and within patients’ tumors for the modeled mutations. A physiologically-based pharmacokinetics model of a TKI drug, integrating a mechanistic modeling of the drug’s mechanism of action, is connected to the disease model. Results After simulating a clinical trial on a virtual population in the jinko platform using the developed model, we were able to follow the proliferating phenotype of each cancer subclone over time, as well as the overall evolution of the tumor size of each patient. The model reproduced the emergence of treatment-resistant subclones on EGFR TKI therapy. Modeling tumor size evolution throughout the clinical trial enabled computing the patients’ time to progression as clinical outcome, based on the RECIST 1.1 criteria, and generating corresponding survival curves. Stratification of virtual patients according to their tumor mutations allowed us to pinpoint key mutations that negatively impacted treatment response. Conclusion Modeling and simulation can help understanding how intratumor heterogeneity affects cancer evolution and drives resistance to treatment. Clinical trial simulations using knowledge-based models of disease and treatment provide relevant additional tools to help clinicians in exploring new hypotheses, providing treatment guidance and supporting therapeutic innovation. References [1] Dearden et al., AnnOnc, 2013 [2] Zhao et al., Sci Rep, 2017 [3] Palgen et al., ActaBiotheor., 2022 Citation Format: Perrine Masson, Claire Couty, Arnaud Nativel, Evgueni Jacob, Raphaël Toueg, Michaël Duruisseaux, Adèle L'Hostis, Jean-Louis Palgen, Claudio Monteiro. Simulations of tumor heterogeneity impact on treatment response using a mechanistic model of EGFR-mutant lung adenocarcinoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 859.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.