Belayer: Modeling discrete and continuous spatial variation in gene expression from spatially resolved transcriptomics

Abstract

Spatially resolved transcriptomics (SRT) technologies simultaneously measure gene expression and spatial location of cells in a 2D tissue slice, enabling the study of spatial patterns of gene expression. Current approaches to model spatial variation in gene expression assume either that gene expression is determined by discrete cell types or that gene expression varies continuously across a tissue slice. However, neither of these modeling assumptions adequately represent spatial variation in gene expression: the first assumption ignores continuous variation within a spatial region containing cells of the same type, while the second assumption does not allow for discontinuous changes in expression, e.g., due to a sharp change in cell type composition. We propose a model of spatial patterns in gene expression that incorporates both discontinuous and continuous spatial variation in gene expression. Specifically, we model the expression of a gene in a layered tissue slice as a piecewise linear function of a single spatial coordinate with potential discontinuities at tissue layer boundaries. We formulate the problem of inferring tissue layer boundaries and gene expression functions for all genes simultaneously. We derive a dynamic programming algorithm to find the optimal boundaries of tissue layers when these boundaries are lines parallel to a coordinate axis. We generalize this algorithm to arbitrarily curved tissue layers by transforming the tissue geometry to be axis-aligned using the theory of conformal maps from complex analysis. We implement these algorithms in a method called Belayer. Applying Belayer to simulated data and to spatial transcriptomics data from the human dorsolateral prefrontal cortex, we demonstrate that Belayer achieves both higher accuracy in clustering tissue layers compared to state-of-the-art SRT clustering methods, and higher accuracy in identifying cortical layer marker genes compared to commonly-used methods for identification of spatially varying genes.

Spateo: multidimensional spatiotemporal modeling of single-cell spatial transcriptomics

“…We also detected Coch, Rorb, Man1a, Npnt, Scnn1a at the lateral, dorsal and medial columns, respectively, consistent with the known order of these markers for cortical functional areas (auditory area, somatosensory area, motor area, retrosplenial area) (Weed et al, 2019) ( Fig. 2k ); these column-specific genes are difficult to identify with other spatially varying gene detection methods (Ma et al, 2022).…”