Integrative spatial analysis of cell morphologies and transcriptional states with MUSE

Abstract

Spatial transcriptomics enables the simultaneous measurement of morphological features and transcriptional profiles of the same cells or regions in tissues.Here we present multi-modal structured embedding (MUSE), an approach to characterize cells and tissue regions by integrating morphological and spatially resolved transcriptional data. We demonstrate that MUSE can discover tissue subpopulations missed by either modality as well as compensate for modality-specific noise. We apply MUSE to diverse datasets containing spatial transcriptomics (seqFISH+, STARmap or Visium) and imaging (hematoxylin and eosin or fluorescence microscopy) modalities. MUSE identified biologically meaningful tissue subpopulations and stereotyped spatial patterning in healthy brain cortex and intestinal tissues. In diseased tissues, MUSE revealed gene biomarkers for proximity to tumor region and heterogeneity of amyloid precursor protein processing across Alzheimer brain regions. MUSE enables the integration of multi-modal data to provide insights into the states, functions and organization of cells in complex biological tissues.

Network models for bridging denoising and identifying spatial domains of spatially resolved transcriptomics

“…We first investigate performance of stACN for denoising by comparing with the typical stand-alone algorithms, such as Sprod, MIST and DIST, with the simulated data [ 61 ], where two factors are involved, i.e., the number of clusters and noise level (Table B in S1 Text ). stACN achieves the best performance as the number of clusters increases from 6 to 15 (Fig J in S1 Text ), demonstrating that stACN is insensitive to perturbation of the number of clusters.…”

Network models for bridging denoising and identifying spatial domains of spatially resolved transcriptomics

“…We first investigate performance of stACN for denoising by comparing with the typical stand-alone algorithms, such as Sprod, MIST and DIST, with the simulated data [ 61 ], where two factors are involved, i.e., the number of clusters and noise level (Table B in S1 Text ). stACN achieves the best performance as the number of clusters increases from 6 to 15 (Fig J in S1 Text ), demonstrating that stACN is insensitive to perturbation of the number of clusters.…”

HESTIA: Scalable Multimodal Integration of Histology and High-Resolution Spatial Transcriptomics for Robust Spatial Domain Identification

“…We evaluated the computational efficiency of HESTIA alongside eight other multimodal methods: ConGI 16 , ConGR 8 , EfNST 17 , MISO 18 , MUSE 7 , SpaGCN 6 , StereoMM 9 , and stLearn 19 . For this benchmark, we used a high-resolution Stereo-seq mouse brain dataset.…”

H2O: A Foundation Model Bridging Histopathology to Spatial Multi-Omics Profiling

“…We next sought to assess how the joint embeddings of H2O improve the cell clustering task. Inspired by the MUSE [54] framework, we evaluated multimodal latent spaces by categorizing clusters as reproduced, newly discovered, or regrouped relative to unimodal results. Specifically, we compared Leiden clusters derived from image features alone with those obtained from the joint latent space provided by H2O (Fig.…”

Network models for bridging denoising and identifying spatial domains of spatially resolved transcriptomics

“…We first investigate performance of stACN for denoising by comparing with the typical stand-alone algorithms, such as Sprod, MIST and DIST, with the simulated data [ 61 ], where two factors are involved, i.e., the number of clusters and noise level (Table B in S1 Text ). stACN achieves the best performance as the number of clusters increases from 6 to 15 (Fig J in S1 Text ), demonstrating that stACN is insensitive to perturbation of the number of clusters.…”

HESTIA: Scalable Multimodal Integration of Histology and High-Resolution Spatial Transcriptomics for Robust Spatial Domain Identification

“…We evaluated the computational efficiency of HESTIA alongside eight other multimodal methods: ConGI 16 , ConGR 8 , EfNST 17 , MISO 18 , MUSE 7 , SpaGCN 6 , StereoMM 9 , and stLearn 19 . For this benchmark, we used a high-resolution Stereo-seq mouse brain dataset.…”

H2O: A Foundation Model Bridging Histopathology to Spatial Multi-Omics Profiling

“…We next sought to assess how the joint embeddings of H2O improve the cell clustering task. Inspired by the MUSE [54] framework, we evaluated multimodal latent spaces by categorizing clusters as reproduced, newly discovered, or regrouped relative to unimodal results. Specifically, we compared Leiden clusters derived from image features alone with those obtained from the joint latent space provided by H2O (Fig.…”

Network models for bridging denoising and identifying spatial domains of spatially resolved transcriptomics

“…We first investigate performance of stACN for denoising by comparing with the typical stand-alone algorithms, such as Sprod, MIST and DIST, with the simulated data [ 61 ], where two factors are involved, i.e., the number of clusters and noise level (Table B in S1 Text ). stACN achieves the best performance as the number of clusters increases from 6 to 15 (Fig J in S1 Text ), demonstrating that stACN is insensitive to perturbation of the number of clusters.…”

HESTIA: Scalable Multimodal Integration of Histology and High-Resolution Spatial Transcriptomics for Robust Spatial Domain Identification

“…We evaluated the computational efficiency of HESTIA alongside eight other multimodal methods: ConGI 16 , ConGR 8 , EfNST 17 , MISO 18 , MUSE 7 , SpaGCN 6 , StereoMM 9 , and stLearn 19 . For this benchmark, we used a high-resolution Stereo-seq mouse brain dataset.…”

H2O: A Foundation Model Bridging Histopathology to Spatial Multi-Omics Profiling

“…We next sought to assess how the joint embeddings of H2O improve the cell clustering task. Inspired by the MUSE [54] framework, we evaluated multimodal latent spaces by categorizing clusters as reproduced, newly discovered, or regrouped relative to unimodal results. Specifically, we compared Leiden clusters derived from image features alone with those obtained from the joint latent space provided by H2O (Fig.…”