Technological advances allow for assaying multiplexed spatially resolved RNA and protein expression profiling of individual cells, thereby capturing physiological tissue contexts of single cell variation. While methods for assaying spatial expression profiles are increasingly accessible, there is a lack of computational approaches that allow for studying the relevance of the spatial organization of tissues on cellcell heterogeneity. Here, we present spatial variance component analysis (SVCA), a computational framework for the analysis of spatial molecular data. SVCA estimates signatures of spatial variance components , thereby quantifying the effect of cellcell interactions, as well as environmental and intrinsic cell features on the expression levels of individual molecules. In application to a breast cancer Imaging Mass Cytometry dataset, our model yields robust spatial variance signatures, identifying cellcell interactions as a major driver of expression heterogeneity. We also apply SVCA to highdimensional imagingderived RNA data, where we identify molecular pathways that are linked to cellcell interactions.