SiGra: Single-cell spatial elucidation through image-augmented graph transformer

Abstract

The recent advances in high throughput molecular imaging push the spatial transcriptomics technologies to the subcellular resolution, which breaks the limitations of both single cell RNA seq and array based spatial profiling. The latest released single cell spatial transcriptomics data from NanoString CosMx and MERSCOPE platforms contains multi channel immunohistochemistry images with rich information of cell types, functions, and morphologies of cellular compartments. In this work, we developed a novel method, Single cell spatial elucidation through image augmented Graph transformer (SiGra), to reveal spatial domains and enhance the substantially sparse and noisy transcriptomics data. SiGra applies hybrid graph transformers over a spatial graph that comprises high-content images and gene expressions of individual cells. SiGra outperformed state of the art methods on both single cell spatial profiles and spot level spatial transcriptomics data from complex tissues. The inclusion of immunohistochemistry images improved the model performance by 37% (95%CI: 27% to 50%). SiGra improves the characterization of intratumor heterogeneity and intercellular communications in human lung cancer samples, meanwhile recovers the known microscopic anatomy in both human brain and mouse liver tissues. Overall, SiGra effectively integrates different spatial modality data to gain deep insights into the spatial cellular ecosystems.

Novel research and future prospects of artificial intelligence in cancer diagnosis and treatment

“…In this way, an image is digitized. Transformer neural networks have recently replaced convolutional neural networks (CNNs) in many nonclinical and clinical image processing jobs because of their enhanced reliability and efficiency in computer vision tasks [17,18]. According to a previous study [19], transformer-based methods outperformed attention-based MIL techniques in terms of data efficiency since they were better at learning from tiny quantities of data.…”

Exploring the Mosaic-like Tissue Architecture of Kidney Diseases Using Relation Equivariant Graph Neural Networks on Spatially Resolved Transcriptomics

“…The majority of cell types in the spot of the samples are annotated as epithelial cells, endothelial cells, immune cells, and stromal cells by experienced nephrology physicians from KPMP. These annotations are utilized as the gold standard benchmarks to test the performance of REGNN and other existing methods, including BayesSpace [12], Giotto [13], SpaGCN [15], RESEPT [16], and SiGra [18]. We used five criteria to quantify the efficacy of these SRT analysis tools, including Adjusted Rand Index (ARI), Rand Index (RI), Normalized Mutual Info score (NMI), Fowlkes Mallows Index (FMI), and Davies-Bouldin Index (DBI) [32].…”

“…RESEPT [16] adopts graph neural networks to learn low dimensional embeddings as RGB images and uses the ResNet50 [17] model to process image segmentation as cell types. Besides gene expression information, SpaGCN [15] and SiGra [18] adopt the H&E images as additional information to improve the model performances. However, there are still huge gaps between existing computational methods and the need to robustly analyze, identify, and annotate diverse cell types in heterogeneous tissue structures, especially in kidney research.…”

Insitutype: likelihood-based cell typing for single cell spatial transcriptomics

“…It scales to large datasets that are intractable for some methods, it gains statistical power by modelling the likelihood of cells’ full expression profiles, it can identify new clusters alongside reference cell types, and it alone harnesses data from cell images. Thus far the cell images accompanying spatial transcriptomics data have only been used for imputation and spatial clustering, not as part of cell typing (Tang 2022).…”