Canonical correlation analysis of high-dimensional data with very small sample support

Abstract

Abstract-This paper is concerned with the analysis of correlation between two high-dimensional data sets when there are only few correlated signal components but the number of samples is very small, possibly much smaller than the dimensions of the data. In such a scenario, a principal component analysis (PCA) rank-reduction preprocessing step is commonly performed before applying canonical correlation analysis (CCA). We present simple, yet very effective approaches to the joint model-order selection of the number of dimensions that should be retained through the PCA step and the number of correlated signals. These approaches are based on reduced-rank versions of the Bartlett-Lawley hypothesis test and the minimum description length information-theoretic criterion. Simulation results show that the techniques perform well for very small sample sizes even in colored noise.

Determining the Dimension of the Improper Signal Subspace in Complex-Valued Data

“…The optimal rank is the one that includes all the improper signal components, but not more than that. "Detector 3" in [10] allows us to jointly choose the optimum rank r and estimate the number d of improper components. The decision rule for d is 1d = max r=1,...,rmax arg min s=0,...,r−1…”

“…Because the min-step will not overfit, we can simply take the maximum over all r from 1 to r max . Here, r max is the maximum allowable rank and is chosen to be sufficiently smaller than M (typically M/3) [10]. This is a much more relaxed condition than requiring m to be sufficiently smaller than M .…”

“…where T (s, r) is the threshold chosen to maintain a specified probability of false alarm P fa , which can be obtained from the χ 2 -approximation. This is "Detector 1" from [10] specialized to the case of detecting the number of correlated components between x and x * . The motivation behind it is similar to the that of (14).…”

Detecting the dimension of the subspace correlated across multiple data sets in the sample poor regime

“…The number of signals correlated across all data sets is denoted by d all , i.e., d all = |{i|ρ (i) pq = 0 ∀p, q|. Prior works have either focused on determining d pq for two data sets [15] or on estimating only d all for more than two data sets [16]- [20]. However, as discussed earlier, even knowing both d pq and d all is not enough to completely determine the underlying correlation structure (except with very special types of correlation structures).…”

“…Hence, d all = 0, d = 4 and d pq is the number of nonzero entries in the corresponding column. In both these examples, the techniques in[15]-[20] provide solutions for either d pq ρ Example with the correlation structure inFig. 1with three data sets each with five signal components.…”