Integrating spatial gene expression and breast tumour morphology via deep learning

He, Bryan; Bergenstråhle, Ludvig; Stenbeck, Linnea; Abid, Abubakar; Andersson, Alma; Borg, Åke; Maaskola, Jonas; Lundeberg, Joakim; Zou, James

doi:10.1038/s41551-020-0578-x

Cited by 281 publications

(348 citation statements)

References 30 publications

Supporting

Mentioning

293

Contrasting

Order By: Relevance

“…In colon histology, we distilled information across WSI to better quantitate how the intermingling of different tissue sub-compartments inform disease stage. These results warrants investigation of other spatially driven processes, such as identifying ROIs correspondent to spatial transcriptomics 40 and integration with high-dimensional omics data types.…”

Section: Discussionmentioning

confidence: 89%

Topological Feature Extraction and Visualization of Whole Slide Images using Graph Neural Networks

Levy

Haudenschild

Barwick³

et al. 2020

Preprint

View full text Add to dashboard Cite

Whole-slide images (WSI) are digitized representations of thin sections of stained tissue from various patient sources (biopsy, resection, exfoliation, fluid) and often exceed 100,000 pixels in any given spatial dimension. Deep learning approaches to digital pathology typically extract information from sub-images (patches) and treat the sub-images as independent entities, ignoring contributing information from vital large-scale architectural relationships. Modeling approaches that can capture higher-order dependencies between neighborhoods of tissue patches have demonstrated the potential to improve predictive accuracy while capturing the most essential slide-level information for prognosis, diagnosis and integration with other omics modalities. Here, we review two promising methods for capturing macro and micro architecture of histology images, Graph Neural Networks, which contextualize patch level information from their neighbors through message passing, and Topological Data Analysis, which distills contextual information into its essential components. We introduce a modeling framework, WSI-GTFE that integrates these two approaches in order to identify and quantify key pathogenic information pathways. To demonstrate a simple use case, we utilize these topological methods to develop a tumor invasion score to stage colon cancer.

show abstract

Section: Discussionmentioning

confidence: 89%

Topological Feature Extraction and Visualization of Whole Slide Images using Graph Neural Networks

Levy

Haudenschild

Barwick³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Using this method, the spatial heterogeneity of MKI67 was identified as an independent predictor of overall survival in breast cancer patients 32 . Realizing the importance of spatial context, new technologies for spatial transcriptomics have begun to emerge and are being increasingly used by the scientific community alongside other methods to spatially resolve molecular measurements 34 – 38 . Recently, spatial transcriptomics data collected from 23 breast cancer patients was used to train a deep neural network to predict spatial variation in gene expression 34 .…”

Section: Introductionmentioning

confidence: 99%

“…Realizing the importance of spatial context, new technologies for spatial transcriptomics have begun to emerge and are being increasingly used by the scientific community alongside other methods to spatially resolve molecular measurements 34 – 38 . Recently, spatial transcriptomics data collected from 23 breast cancer patients was used to train a deep neural network to predict spatial variation in gene expression 34 . In a cohort of 41 gastric cancer patients, an association between heterogeneity and survival was discovered using a genome-wide single-nucleotide variation array to estimate the number of clones 21 .…”

Section: Introductionmentioning

confidence: 99%

Spatial transcriptomics inferred from pathology whole-slide images links tumor heterogeneity to survival in breast and lung cancer

Levy-Jurgenson

Tekpli

Kristensen

et al. 2020

Sci Rep

108

View full text Add to dashboard Cite

Digital analysis of pathology whole-slide images is fast becoming a game changer in cancer diagnosis and treatment. Specifically, deep learning methods have shown great potential to support pathology analysis, with recent studies identifying molecular traits that were not previously recognized in pathology H&E whole-slide images. Simultaneous to these developments, it is becoming increasingly evident that tumor heterogeneity is an important determinant of cancer prognosis and susceptibility to treatment, and should therefore play a role in the evolving practices of matching treatment protocols to patients. State of the art diagnostic procedures, however, do not provide automated methods for characterizing and/or quantifying tumor heterogeneity, certainly not in a spatial context. Further, existing methods for analyzing pathology whole-slide images from bulk measurements require many training samples and complex pipelines. Our work addresses these two challenges. First, we train deep learning models to spatially resolve bulk mRNA and miRNA expression levels on pathology whole-slide images (WSIs). Our models reach up to 0.95 AUC on held-out test sets from two cancer cohorts using a simple training pipeline and a small number of training samples. Using the inferred gene expression levels, we further develop a method to spatially characterize tumor heterogeneity. Specifically, we produce tumor molecular cartographies and heterogeneity maps of WSIs and formulate a heterogeneity index (HTI) that quantifies the level of heterogeneity within these maps. Applying our methods to breast and lung cancer slides, we show a significant statistical link between heterogeneity and survival. Our methods potentially open a new and accessible approach to investigating tumor heterogeneity and other spatial molecular properties and their link to clinical characteristics, including treatment susceptibility and survival.

show abstract

“…Recent work on analysis of morphological states of cells has relied on images of fixed cells labeled with a panel of fluorescent markers (1), live three-dimensional imaging of the membrane labeled with genetic markers (2), and phase contrast imaging of live cells (3)(4)(5)(6). The morphological states have been analyzed with low dimensional representations computed with geometric or biophysical models (3,(7)(8)(9)(10)(11), supervised learning of morphological labels (4,(12)(13)(14)(15)(16)(17), and, recently, self-supervised learning of latent representations of morphology (5,6). These analytical approaches have been inspired by the need for quantitative descriptions of specific, complex biological functions, such as motility of single cells (2,3,7,8,18), collective cell migration (9,11), cell cycle (4,12,13), spatial gene expression (17), and spatial protein expression (14,16).In addition, data-driven integration of the morphology and gene expression (13,17,(19)(20)(21)(22) is now enabling rapid analysis of functional roles of genes.…”

Section: Introductionmentioning

confidence: 99%

DynaMorph: self-supervised learning of morphodynamic states of live cells

Chhun

Попова³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Morphological states of human cells are widely imaged and analyzed to diagnose diseases and to discover biological mechanisms. Morphodynamics of cells capture their functions more fully than their morphology. Discovery of morphodynamic states of human cells is challenging, because genetic labeling or manual annotation may not be feasible. We propose a computational framework, DynaMorph, that combines quantitative label-free imaging and deep learning for automated discovery of morphodynamic states. As a case study, we apply DynaMorph to study the morphodynamic states of live primary human microglia, which are mobile immune cells of the brain that exhibit complex functional states. DynaMorph identifies two distinct morphodynamic states of microglia under perturbation by cytokines and glioblastoma supernatant. We find that microglia actively transition between the two states. Moreover, single-cell RNA-sequencing of the perturbed microglia shows that the morphodynamic states correspond to distinct transcriptomic clusters of the cells, revealing how perturbations alter gene expression and phenotype. DynaMorph can broadly enable automated discovery of functional states of cellular systems.

show abstract

Integrating spatial gene expression and breast tumour morphology via deep learning

Cited by 281 publications

References 30 publications

Topological Feature Extraction and Visualization of Whole Slide Images using Graph Neural Networks

Topological Feature Extraction and Visualization of Whole Slide Images using Graph Neural Networks

Spatial transcriptomics inferred from pathology whole-slide images links tumor heterogeneity to survival in breast and lung cancer

DynaMorph: self-supervised learning of morphodynamic states of live cells

Contact Info

Product

Resources

About