Mixed Effects Machine Learning Models for Colon Cancer Metastasis Prediction using Spatially Localized Immuno-Oncology Markers

Levy, Joshua; Bobak, Carly A.; Nasir-Moin, Mustafa; Veziroglu, Eren M.; Palisoul, Scott; Barney, Rachael E.; Salas, Lucas A.; Christensen, Brock C.; Tsongalis, Gregory J.; Vaickus, Louis

doi:10.1101/2021.08.01.454649

Cited by 3 publications

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Development of Biologically Interpretable Multimodal Deep Learning Model for Cancer Prognosis Prediction

Azher

Vaickus

Salas

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Robust cancer prognostication can enable more effective patient care and management, which may potentially improve health outcomes. Deep learning has proven to be a powerful tool to extract meaningful information from cancer patient data. In recent years it has displayed promise in quantifying prognostication by predicting patient risk. However, most current deep learning-based cancer prognosis prediction methods use only a single data source and miss out on learning from potentially rich relationships across modalities. Existing multimodal approaches are challenging to interpret in a biological or medical context, limiting real-world clinical integration as a trustworthy prognostic decision aid. Here, we developed a multimodal modeling approach that can integrate information from the central modalities of gene expression, DNA methylation, and histopathological imaging with clinical information for cancer prognosis prediction. Our multimodal modeling approach combines pathway and gene-based sparsely coded layers with patch-based graph convolutional networks to facilitate biological interpretation of the model results. We present a preliminary analysis that compares the potential applicability of combining all modalities to uni- or bi-modal approaches. Leveraging data from four cancer subtypes from the Cancer Genome Atlas, results demonstrate the encouraging performance of our multimodal approach (C-index=0.660 without clinical features; C-index=0.665 with clinical features) across four cancer subtypes versus unimodal approaches and existing state-of-the-art approaches. This work brings insight to the development of interpretable multimodal methods of applying AI to biomedical data and can potentially serve as a foundation for clinical implementations of such software. We plan to follow up this preliminary analysis with an in-depth exploration of factors to improve multimodal modeling approaches on an in-house dataset.

show abstract

Development of Biologically Interpretable Multimodal Deep Learning Model for Cancer Prognosis Prediction

Azher

Vaickus

Salas

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Inferring Spatially Resolved Transcriptomics Data from Whole Slide Images for the Assessment of Colorectal Tumor Metastasis: A Feasibility Study

Fatemi

Feng²,

Sharma

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Over 150,000 Americans are diagnosed with colorectal cancer (CRC) every year, and annually over 50,000 individuals will die from CRC, necessitating improvements in screening, prognostication, disease management, and therapeutic options. Tumor metastasis is the primary factor related to the risk of recurrence and mortality. Yet, screening for nodal and distant metastasis is costly, and invasive and incomplete resection may hamper adequate assessment. Signatures of the tumor-immune microenvironment (TIME) at the primary site can provide valuable insights into the aggressiveness of the tumor and the effectiveness of various treatment options. Spatially-resolved transcriptomics technologies offer an unprecedented characterization of TIME through high multiplexing, yet their scope is constrained by cost. Meanwhile, it has long been suspected that histological, cytological and macroarchitectural tissue characteristics correlate well with molecular information (e.g., gene expression). Thus, a method for predicting transcriptomics data through inference of RNA patterns from whole slide images (WSI) is a key step in studying metastasis at scale. In this work, we collected and preprocessed Visium spatial transcriptomics data (17,943 genes at up to 5,000 spots per patient sampled in a honeycomb pattern) from tissue across four stage-III matched colorectal cancer patients. We compare and prototype several convolutional, Transformer, and graph convolutional neural networks to predict spatial RNA patterns under the hypothesis that the transformer and graph-based approaches better capture relevant spatial tissue architecture. We further analyzed the model’s ability to recapitulate spatial autocorrelation statistics using SPARK and SpatialDE. Overall, results indicate that the transformer and graph-based approaches were unable to outperform the convolutional neural network architecture, though they exhibited optimal performance for relevant disease-associated genes. Initial findings suggest that different neural networks that operate on different scales are relevant for capturing distinct disease pathways (e.g., epithelial to mesenchymal transition). We add further evidence that deep learning models can accurately predict gene expression in whole slide images and comment on understudied factors which may increase its external applicability (e.g., tissue context). Our preliminary work will motivate further investigation of inference for molecular patterns from whole slide images as metastasis predictors and in other applications.

show abstract

Development of biologically interpretable multimodal deep learning model for cancer prognosis prediction

Azher

Vaickus

Salas

et al. 2022

Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing

View full text Add to dashboard Cite

Mixed Effects Machine Learning Models for Colon Cancer Metastasis Prediction using Spatially Localized Immuno-Oncology Markers

Cited by 3 publications

References 41 publications

Development of Biologically Interpretable Multimodal Deep Learning Model for Cancer Prognosis Prediction

Development of Biologically Interpretable Multimodal Deep Learning Model for Cancer Prognosis Prediction

Inferring Spatially Resolved Transcriptomics Data from Whole Slide Images for the Assessment of Colorectal Tumor Metastasis: A Feasibility Study

Development of biologically interpretable multimodal deep learning model for cancer prognosis prediction

Contact Info

Product

Resources

About