Field Theoretical Approach for Signal Detection in Nearly Continuous Positive Spectra I: Matricial Data

Abstract

Renormalization group techniques are widely used in modern physics to describe the relevant low energy aspects of systems involving a large number of degrees of freedom. Those techniques are thus expected to be a powerful tool to address open issues in data analysis when datasets are highly correlated. Signal detection and recognition for a covariance matrix having a nearly continuous spectra is currently one of these opened issues. First, investigations in this direction have been proposed in recent investigations from an analogy between coarse-graining and principal component analysis (PCA), regarding separation of sampling noise modes as a UV cut-off for small eigenvalues of the covariance matrix. The field theoretical framework proposed in this paper is a synthesis of these complementary point of views, aiming to be a general and operational framework, both for theoretical investigations and for experimental detection. Our investigations focus on signal detection. They exhibit numerical investigations in favor of a connection between symmetry breaking and the existence of an intrinsic detection threshold.

Signal Detection in Nearly Continuous Spectra and ℤ2-Symmetry Breaking

“…Following the non-perturbative framework introduced in [14,15], we have shown in this study that this new way of detecting a signal in such continuous spectra is related to a symmetry-breaking phenomenon. Note that this phase transition finding is in the same vein as those well studied in the spiked model associated with the PCA [51][52][53][54].…”

“…In this investigation, our work follows a fruitful path for the use of RG in a context connected with PCA [14,15,17]. Namely, the challenging context studied in this series of works tries to address, with the lens of RG, the problem of signal detection in a continuous spectrum where the distinction between signal and noise appears to be arbitrary with the standard PCA.…”

“…This is in particular the case within the big data paradigm. Principal component analysis (PCA) (see [14][15][16][17][18][19][20][21][22][23][24][25][26][27][28]) looks for a linear projection into a lower-dimensional space, keeping only the relevant features, which is exactly what the RG aims to do. For this reason, the RG is expected to be a relevant and competitive approach to standard PCA, and perhaps a solid alternative in context where PCA fails.…”

“…For this reason, RG is expected to be a relevant and competitive approach to standard PCA. This can be achieved through a field theoretical embedding, as considered in [15,16], from an analogy with what happens in standard field theory. This field theory can be viewed as describing some unconventional "matter" filling an abstract space of dimension 1.…”

“…In this paper, we only claim to build an effective theory, in the same class of long-range equivalence as the "true" theory (see Figure 1). This point of view was the one developed in [14][15][16][17]. In these papers, the authors were able to characterize the presence of a signal, and estimate the breaking point between signal and noise, by the fact that the first non-Gaussian perturbation, which is relevant for a purely MP distribution, becomes irrelevant for a sufficiently strong signal.…”