Development of Biologically Interpretable Multimodal Deep Learning Model for Cancer Prognosis Prediction

Abstract

Robust cancer prognostication can enable more effective patient care and management, which may potentially improve health outcomes. Deep learning has proven to be a powerful tool to extract meaningful information from cancer patient data. In recent years it has displayed promise in quantifying prognostication by predicting patient risk. However, most current deep learning-based cancer prognosis prediction methods use only a single data source and miss out on learning from potentially rich relationships across modalities. Existing multimodal approaches are challenging to interpret in a biological or medical context, limiting real-world clinical integration as a trustworthy prognostic decision aid. Here, we developed a multimodal modeling approach that can integrate information from the central modalities of gene expression, DNA methylation, and histopathological imaging with clinical information for cancer prognosis prediction. Our multimodal modeling approach combines pathway and gene-based sparsely coded layers with patch-based graph convolutional networks to facilitate biological interpretation of the model results. We present a preliminary analysis that compares the potential applicability of combining all modalities to uni- or bi-modal approaches. Leveraging data from four cancer subtypes from the Cancer Genome Atlas, results demonstrate the encouraging performance of our multimodal approach (C-index=0.660 without clinical features; C-index=0.665 with clinical features) across four cancer subtypes versus unimodal approaches and existing state-of-the-art approaches. This work brings insight to the development of interpretable multimodal methods of applying AI to biomedical data and can potentially serve as a foundation for clinical implementations of such software. We plan to follow up this preliminary analysis with an in-depth exploration of factors to improve multimodal modeling approaches on an in-house dataset.

Biologically-informed deep neural networks provide quantitative assessment of intratumoral heterogeneity in post-treatment glioblastoma

“…Depending on the form of the domain knowledge, the approaches for integration vary. Some researchers proposed to use the knowledge of biological pathways to guide the architecture design of DL (31, 32). Moreover, in some biomedical fields, domain knowledge exists in the form of algebraic equations representing biological principles which are imposed on ML models (24, 34).…”

“…Second, integration of biological/biomedical domain knowledge, such as, biological principles, empirical models, simulations, and knowledge graphs, can provide a rich source of information (pseudo data) to help alleviate the data shortage in training DL models. Biologically-or biomedically-informed deep learning (BIDL) has been proposed to use domain knowledge to guide the design of DL architecture (31,32), as attribution priors of features (33), to regularize the model predictions, coefficients, or latent feature representations (24,34), and implicitly leveraging knowledge from other domains through transfer learning (35)(36)(37)(38). However, these existing works lack the capability of integrating hierarchical domain knowledge which is hard to be described in mathematical formulation.…”

Assessment of Emerging Pretraining Strategies in Interpretable Multimodal Deep Learning for Cancer Prognostication

“…Clinical or biological interpretability of model decision making can foster practitioner and patient trust in the technology, which is imperative for the adoption of ML technologies in the medical context [12,13]. Several recent multimodal cancer prognostication studies have applied specific architectures and techniques to enhance model interpretability and offer explanations for model predictions [14,15]. For instance, specialized encoding schemes which reflect prior biological structure (e.g., genes to pathways) offer opportunities to reduce noise and further increase the capacity to interrogate the model findings.…”

“…Graph neural networks (GNNs) -where nodes are representations/embeddings of patches and edge connections are formed based on spatial adjacency -are promising modeling approaches based on their ability to capture complex micro and macro architectural relationships in WSI based on spatial connectivity [19]. Few applications of GNNs to WSI for cancer prognostication currently exist [14,20]. Further investigation of additional cancer types and experimental setups could prove beneficial.…”

A Deep Survival EWAS approach estimating risk profile based on pre-diagnostic DNA methylation: An application to breast cancer time to diagnosis

“…Indeed, NN are optimized to extract rich latent features from DNAm data, handling multi-collinearity, noise, and considering the complex non-linear interactions between very large amounts of input covariates [10]. Quite a few examples exist of DL-based methods for patients classification [11][12][13], risk prediction and survival modeling [14][15][16][17] from DNAm data. On top of these, some recent works demonstrated the usefulness of AutoEncoders (AE), Variational AE (VAE) and DL models specifically to obtain DL-based EWAS (henceforth, Deep EWAS) [11,12,18], i.e.…”