Comparison of statistical pattern-recognition algorithms for hybrid processing II Eigenvector-based algorithm

Abstract

The pattern-recognition algorithms based on eigenvector analysis (group 2) are theoretically and experimentally compared. Group 2 consists of Foley-Sammon (F-S) transform, Hotelling trace criterion (HTC), Fukunaga-Koontz (F-K) transform, linear discriminant function (LDF), and generalized matched filter (GMF) algorithms. It is shown that all eigenvector-based algorithms can be represented in a generalized eigenvector form. However, the calculations of the discriminant vectors are different for different algorithms. Summaries of methods of calculating the discriminant functions for the F-S, HTC, and F-K transforms are provided. Especially for the more practical, underdetermined case, where the number of training images is less than the number of pixels in each image, the calculations usually require the inversion of a large, singular pixel correlation (or covariance) matrix. We suggest solving this problem by finding its pseudoinverse, which requires inverting only the smaller, nonsingular imagecorrection (or covariance) matrix plus multiplying several nonsingular matrices. We also compare theoretically the classification performance with discriminant functions of the F-S, HTC, and F-K with the LDF and GMF algorithms and the linear-mapping-based algorithms with the eigenvector-based algorithms. Experimentally, we compare the eigenvector-based algorithms, using two sets of image data bases with each image consisting of 64 x 64 pixels.

An efficient P300-based brain–computer interface for disabled subjects

“…In particular, this happens when the number of features D becomes larger than the number of training examples N. A simple solution for this problem is to replace the inverse S −1 W by the Moore-Penrose pseudo-inverse S † W (Tian et al, 1988).…”

P300 detection using nonlinear independent component analysis

“…In particular, this happens when the number of features D becomes larger than the number of training examples N. A simple solution for this problem is to replace the inverse 1 W S − by the Moore-Penrose pseudo-inverse + W S [22]. The output of FLDA given an input vector x is simply the product x w Tˆ.…”

Convergence proof of matrix dynamics for online linear discriminant analysis

“…We presume that W is (strictly) positive definite in the present paper; this holds in most cases of real tasks. Regarding problems that arise on singular W , see [32,14,7,35,30].…”