The MCA EXIN neuron for the minor component analysis

Abstract

The minor component analysis (MCA) deals with the recovery of the eigenvector associated to the smallest eigenvalue of the autocorrelation matrix of the input data and is a very important tool for signal processing and data analysis. It is almost exclusively solved by linear neurons. This paper presents a linear neuron endowed with a novel learning law, called MCA EXINn and analyzes its features. The neural literature about MCA is very poor, in the sense that both a little theoretical basis is given (almost always focusing on the ODE asymptotic approximation) and only experiments on toy problems (at most four-dimensional problems) are presented, without any numerical analysis. This work addresses these problems and lays sound theoretical foundations for the neural MCA theory. In particular, it classifies the MCA neurons according to the Riemannian metric and justifies, from the analysis of the degeneracy of the error cost; the different behavior in approaching convergence. The cost landscape is studied and used as a basis for the analysis of the asymptotic behavior. All the phases of the dynamics of the MCA algorithms are investigated in detail and, together with the numerical analysis, lead to the identification of three possible kinds of divergence, here called sudden, dynamic, and numerical. The importance of the choice of low initial conditions is also explained. A lot of importance is given to the experimental part, where simulations on high-dimensional problems are,presented and analyzed. The orthogonal regression or total least squares (TLS) technique is also presented, together with a real-world application on the identification of the parameters of an electrical machine. It can be concluded that MCA EXIN is the best MCA neuron in terms of stability (no finite time divergence), speed, and accuracy.

Dynamical Systems for Joint Principal and Minor Component Analysis

“…A well-known tools for computing the principal and minor subspace of a data matrix is Oja's rule [1,2] and several variations of it [3,4,5,6]. Many other methods that involve Rayleigh and inverse Rayleigh quotients [7,8,9] are derived for performing principal subspace analysis (PSA), and minor subspace analysis (MSA), principal and minor component analysis (PCA, and MCA, respectively).…”

A Self-Stabilizing Learning Rule for Minor Component Analysis

“…This paper reviews single-neuron MCA rules. The review is based on the study by Cirrincione et al 2 , where the rules OJAn, 3 , LUO, 4,5,6 MCA-EXIN (here EXIN for short), 2 OJA+, 8 and FENG…”

“…MCA has several applications in adaptive signal processing, parameter estimation, and computer vision. 1,2 In neural network techniques for MCA, estimates of the minor eigenvectors are updated by sequentially processing data vectors drawn from the distribution. Neural MCA methods are advantageous for highdimensional data, since they avoid the computation of the covariance matrix, and are suitable for the tracking of non-stationary distributions.…”

“…Cirrincione et al 2 distinguish between two classes of learning rules according to the time course of the length of the eigenvector estimate. In the rules of the first class, OJAn, LUO, and EXIN, the length of the weight vector in the fixed point is undetermined.…”

“…However, when a numerical procedure (like Euler's method) is applied, all these rules are plagued by "sudden divergence" of the weight vector length. 7,2 In the rules of the second class, DOUG, CHEN, XU*, OJA*, OJA+, FENG, AMEX, ZLlog, ZLdiff, and WANG, the weight vector length converges towards a fixed value.…”