Deep learning-based survival prediction for multiple cancer types using histopathology images

Abstract

Providing prognostic information at the time of cancer diagnosis has important implications for treatment and monitoring. Although cancer staging, histopathological assessment, molecular features, and clinical variables can provide useful prognostic insights, improving risk stratification remains an active research area. We developed a deep learning system (DLS) to predict disease specific survival across 10 cancer types from The Cancer Genome Atlas (TCGA). We used a weakly-supervised approach without pixel-level annotations, and tested three different survival loss functions. The DLS was developed using 9,086 slides from 3,664 cases and evaluated using 3,009 slides from 1,216 cases. In multivariable Cox regression analysis of the combined cohort including all 10 cancers, the DLS was significantly associated with disease specific survival (hazard ratio of 1.58, 95% CI 1.28-1.70, p<0.0001) after adjusting for cancer type, stage, age, and sex. In a per-cancer adjusted subanalysis, the DLS remained a significant predictor of survival in 5 of 10 cancer types. Compared to a baseline model including stage, age, and sex, the c-index of the model demonstrated an absolute 3.7% improvement (95% CI 1.0-6.5) in the combined cohort. Additionally, our models stratified patients within individual cancer stages, particularly stage II (p = 0.025) and stage III (p<0.001). By developing and evaluating prognostic models across multiple cancer types, this work represents one of the most comprehensive studies exploring the direct prediction of clinical outcomes using deep learning and histopathology images. Our analysis demonstrates the potential for this approach to provide significant prognostic information in multiple cancer types, and even within specific pathologic stages. However, given the relatively small number of cases and observed clinical events for a deep learning task of this type, we observed wide confidence intervals for model performance, thus highlighting that future work will benefit from larger datasets assembled for the purposes for survival modeling.

Predicting Prognosis and IDH Mutation Status for Patients with Lower-Grade Gliomas Using Whole Slide Images

“…For the prognosis prediction task, our model architecture is adapted from the proposed work by Wulczyn et al 24 and is illustrated in Figure 3. In summary, for each batch, n participants were randomly chosen from the training dataset.…”

Deep learning-enabled medical computer vision

“…International multi-institutional evaluation is a robust method to determine generalizability of models across diverse populations, medical equipment, resource settings, and practice patterns. In addition, using multi-task learning 147 to train models to perform a variety of tasks rather than one narrowly defined task, such as multi-cancer detection from histopathology images 148 , makes them more generally useful and often more robust.…”

The Impact of Digital Histopathology Batch Effect on Deep Learning Model Accuracy and Bias

“…Studies demonstrating histologic discrimination of survival and recurrence in glioblastoma 41,43,44 , renal cell cancer 41 , and lung cancer 45 patients from TCGA which lack external validation cohorts may have biased estimates of outcome. Prediction of survival may also suffer from this bias 46 even when correcting for age, stage, and sex, as other factors that vary by site also contribute to outcome, ranging from ethnicity of enrollees, to the treatment available at academic vs community centers. Given that traditional image and textural features vary between sites in TCGA, it is likely that non-deep learning prognostic studies that predict prognosis from traditional image analysis features may suffer from a similar bias 47 .…”