Graph deep learning for the characterization of tumour microenvironments from spatial protein profiles in tissue specimens

Abstract

Multiplexed immunofluorescence imaging enables high-dimensional molecular profiling at subcellular resolution. However, learning disease-relevant cellular environments from these rich imaging data is an open challenge. We developed SPAtial CEllular Graphical Modeling (SPACE-GM), a geometric deep learning framework that flexibly models tumor microenvironments (TMEs) as cellular graphs. We applied SPACE-GM to 658 head-and-neck and colorectal human cancer samples assayed with 40-plex immunofluorescence imaging to identify spatial motifs associated with cancer recurrence and patient survival after immunotherapy. SPACE-GM is substantially more accurate in predicting patient outcomes than previous approaches for modeling spatial data using neighborhood cell-type compositions. Computational interpretation of the disease-relevant microenvironments identified by SPACE-GM generates insights into the effect of spatial dispersion of tumor cells and granulocytes on patient prognosis.

Graph neural networks learn emergent tissue properties from spatial molecular profiles

“…Graph attention 12 and other graph kernels with more degrees of freedom may be more sensitive to complex tissue niche motives. Second, pooling across the graph may be performed globally or hierarchically, as previously also discussed by Wu et al 5 . Aggregation of information across a graph becomes even more relevant if larger graphs are considered, which may for example become available in the context of tissue clearing 13 .…”

“…3a-d). It was previously reported that the spatial distribution of immune cells in colorectal cancer is predictive of disease outcome and is used to stratify tumors 5,10 . Indeed, we found that an MLP trained on an immune cell dispersion estimate per image was more predictive of tumor class in test images of colorectal cancer than the baseline pseudo-bulk model that only reflects composition but not architecture (Fig.…”

“…This model class captures both molecular information of individual cells as input node feature vectors and the architecture of the tissue through the spatial proximity graph. Previously, such graph neural network architectures have been used to model tumor grades based on H&E images 8 and spatial proteomics [5][6][7] . We consider examples of graph-level supervision on three data sets: Two large cohorts of imaging mass cytometry (IMC) of breast cancer biopsy data stratified by tumor grade and other tumor characteristics (IMC -Jackson 9 with 559 images from 350 patients and IMC -METABRIC 9 with 500 images from 454 patients) and a cohort of CODEX samples of colorectal cancer biopsies stratified by tumor classification and disease outcome (CODEXcolorectal cancer 10 , 140 images from 35 patients).…”

“…Spatial information and graphs of cells extend the capability of such supervised models to not only detect cellular phenotypes that correlate with the labels, but also motifs of tissue architecture, such as cellular niches 4 . Because of their explicit representation of cells as constituent building blocks of a tissue, graph neural networks promise to be more interpretable with respect to niches than convolutional neural networks on tissue images and, indeed, have been recently successfully deployed for tumor phenotype prediction from spatial proteomics data [5][6][7] . Here, we perform a comparative ablation study over spatial features and single-cell resolution for graph neural networks that predict tumor phenotypes from spatial graphs with gene expression or categorical cell type node features.…”

Consensus tissue domain detection in spatial multi-omics data using MILWRM

“…Since this approach focuses directly on pixel-level information instead of reconstruction from single-cell data, it can identify both extracellular structures and cellular communities over a range of microto macro-scale. Pixel-based analysis also forms the basis of modern artificial intelligence learning from imaging, and thus paves the way for more complex learning algorithms to be applied to multiplex tissue data (10, 45,61). Various methods are currently available for pixel-based spatial domain detection from ST data (8, 55,63).…”