Finding Stable Groups of Cross-Correlated Features in Two Data Sets With Common Samples

Abstract

Multi-view data, in which data of different types are obtained from a common set of samples, is now common in many applied scientific problems. An important problem in the analysis of multi-view data is identifying interactions between groups of features from different data types. A bimodule is a pair (A, B) of feature sets from two different data types such that the aggregate cross-correlation between the features in A and those in B is large. A bimodule (A, B) is stable if A coincides with the set of features having significant aggregate correlation with the features in B, and vice-versa. At the population level, stable bimodules correspond to connected components of the cross-correlation network, which is the bipartite graph whose edges are pairs of features with non-zero cross-correlations.We develop and investigate an iterative, testing-based procedure, called BSP, to identify stable bimodules in two moderate-to highdimensional data sets. BSP relies on permutation-based p-values for test statistics equal to sums of squared cross-correlations. These pvalues are approximated using tail probabilities of gamma distributions that are fit using estimates of the permutation moments of the test statistic. Our moment estimates depend on the eigenvalues of the intra-correlation matrices of A and B, and as a result the significance of observed cross-correlations accounts for the correlations within each data type.We carry out a thorough simulation study to assess the performance of BSP, and present an extended application of BSP to the problem of expression quantitative trait loci (eQTL) analysis using recent data from the GTEx project. In addition, we apply BSP to climatology data in order to identify regions in North America where annual temperature variation affects precipitation.The method is available as an R package at https://github.com/ miheerdew/cbce.

The folded concave Laplacian spectral penalty learns block diagonal sparsity patterns with the strong oracle property

“…There is a growing literature on block diagonal estimation including: covariance matrix and graphical model estimation (Marlin and Murphy, 2009;Pavlenko et al, 2012;Tan et al, 2015;Hyodo et al, 2015;Sun et al, 2015;Egilmez et al, 2017;Devijver and Gallopin, 2018;Kumar et al, 2019;Broto et al, 2019), community detection (Nie et al, 2016), co-clustering (Han et al, 2017;Nie et al, 2017), subspace clustering (Feng et al, 2014;Lu et al, 2018), principal components analysis (Asteris et al, 2015), bipartite cross-correlation clustering (Dewaskar et al, 2020), neural network regularization (Tam and Dunson, 2020), and multi-view clustering (Carmichael, 2020).…”