Discriminative Common Vector Method With Kernels

Abstract

In some pattern recognition tasks, the dimension of the sample space is larger than the number of samples in the training set. This is known as the "small sample size problem". Linear discriminant analysis (LDA) techniques cannot be applied directly to the small sample size case. The small sample size problem is also encountered when kernel approaches are used for recognition. In this paper, we attempt to answer the question of "How should one choose the optimal projection vectors for feature extraction in the small sample size case?" Based on our findings, we propose a new method called the kernel discriminative common vector method. In this method, we first nonlinearly map the original input space to an implicit higher dimensional feature space, in which the data are hoped to be linearly separable. Then, the optimal projection vectors are computed in this transformed space. The proposed method yields an optimal solution for maximizing a modified Fisher's linear discriminant criterion, discussed in the paper. Thus, under certain conditions, a 100% recognition rate is guaranteed for the training set samples. Experiments on test data also show that, in many situations, the generalization performance of the proposed method compares favorably with other kernel approaches.

Fusing Gabor and LBP Feature Sets for Kernel-Based Face Recognition

“…A simple Nearest Neighbour (NN) classifier is applied to the resulting KDCV feature vector. H. Cevikalp et al [4] have shown that the combination of KDCV and NN significantly outperforms several other kernel methods including KPCA+LDA and SVM in related problems.…”

“…This is the only point at which class label information is used during training. It is based on a kernelized variant of Linear Discriminant Analysis (LDA) [24] called KDCV [4]. Classical LDA seeks a low-dimensional projection matrix P that maximizes the objective function…”

“…Cevikalp et al [4] proved that the optimal discriminant subspace in the sense of (5) is the intersection of the null space N (S W ) of S W and the range space R(S T ) of S T , and to find it one can first project the training set sample onto N (S W ) and then apply PCA. This method is called Discriminative Common Vectors (DCV) [5] because all of the training samples in a class are projected to a unique vector in N (S W ) called the class' common vector.…”

“…In many face recognition problems the class distributions are not separable using linear DCV but introducing a nonlinear embedding φ : R d → F into a kernel-induced feature space F allows them to be separated. Kernel DCV [4] finds projection vectors that optimize the null space LDA criterion (5) in the induced feature space F by applying KPCA to project the training set onto the range space R(S φ T ) of S φ T , the total scatter matrix induced in F , then finding an orthonormal basis for the null space N (S φ W ) of the within-class scatter matrix S φ W within this range space. The computation is kernalizable (expressible using inner products) precisely because it suffices to work within the span of S φ T : although N (S φ W ) typically contains many directions orthogonal to this, they are irrelevant as far as inter-class discrimination is concerned because test sample components in these directions are identical for all classes and hence not useful for discrimination based on this training set.…”

“…The reduced feature vectors are separately normalized before being concatenated into a single combined feature vector. Finally the Kernel Discriminative Common Vector (KDCV) [4] method is applied to the combined feature vector to provide effective multi-class recognition from relatively few training examples. To illustrate the effectiveness of our approach we present experimental results on three state-of-the-art face recognition datasets containing large lighting variations similar to those encountered in natural images taken under uncontrolled conditions: the Face Recognition Grand Challenge 1.0.4 and 2.0.4 datasets [21] and FERET [22].…”