A new optimization criterion for generalized discriminant analysis on undersampled problems

Abstract

Abstract-An optimization criterion is presented for discriminant analysis. The criterion extends the optimization criteria of the classical Linear Discriminant Analysis (LDA) through the use of the pseudoinverse when the scatter matrices are singular. It is applicable regardless of the relative sizes of the data dimension and sample size, overcoming a limitation of classical LDA. The optimization problem can be solved analytically by applying the Generalized Singular Value Decomposition (GSVD) technique. The pseudoinverse has been suggested and used for undersampled problems in the past, where the data dimension exceeds the number of data points. The criterion proposed in this paper provides a theoretical justification for this procedure. An approximation algorithm for the GSVD-based approach is also presented. It reduces the computational complexity by finding subclusters of each cluster and uses their centroids to capture the structure of each cluster. This reduced problem yields much smaller matrices to which the GSVD can be applied efficiently. Experiments on text data, with up to 7,000 dimensions, show that the approximation algorithm produces results that are close to those produced by the exact algorithm.

Gene Selection for Microarray Data by a LDA-Based Genetic Algorithm

“…For the Ovarian cancer dataset, the LDA-GA gives a prediction accuracy of 98.4% with a subset of only 4 genes. The reference algorithms have a slightly better classification rate, but select much more genes (20,26,75). Notice that a perfect rate is reported in [15] with 50 genes.…”

“…In this case, it is not possible to compute S −1 W . To overcome the singularity problem, recent works have proposed different methods like the null space method [28], orthogonal LDA [26], uncorrelated LDA [27,26] (see also [17] for a comparison of these methods). The two last techniques use the pseudo inverse method to solve the small sample size problem and this is the approach we apply in this work.…”

A computerized system of nail-fold capillaroscopy for dry eye disease diagnosis

“…We will try to upgrade this part in our next version. Also the LDA (Ye 2006;Li et al 2004;Otsu 2005;Shiraki et al 2006) and SIFT-based recognition process (Kisku et al 2007;Geng and Jiang 2009) will be introduced in the next version of the system. A discriminant function could be considered to improve the identification of DED.…”

Alleviating the small sample-size problem in i-vector based speaker verification

“…More recently, null-space LDA [17,18] and orthogonal LDA [19,20] have been proposed to address the small sample-size problem. In null-space LDA, the between-class distance is maximized in the null space of the within-class scatter matrix.…”