Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations

Abstract

We address the problem of comparing sets of images for object recognition, where the sets may represent variations in an object's appearance due to changing camera pose and lighting conditions. Canonical Correlations (also known as principal or canonical angles), which can be thought of as the angles between two d-dimensional subspaces, have recently attracted attention for image set matching. Canonical correlations offer many benefits in accuracy, efficiency, and robustness compared to the two main classical methods: parametric distribution-based and nonparametric sample-based matching of sets. Here, this is first demonstrated experimentally for reasonably sized data sets using existing methods exploiting canonical correlations. Motivated by their proven effectiveness, a novel discriminative learning method over sets is proposed for set classification. Specifically, inspired by classical Linear Discriminant Analysis (LDA), we develop a linear discriminant function that maximizes the canonical correlations of within-class sets and minimizes the canonical correlations of between-class sets. Image sets transformed by the discriminant function are then compared by the canonical correlations. Classical orthogonal subspace method (OSM) is also investigated for the similar purpose and compared with the proposed method. The proposed method is evaluated on various object recognition problems using face image sets with arbitrary motion captured under different illuminations and image sets of 500 general objects taken at different views. The method is also applied to object category recognition using ETH-80 database. The proposed method is shown to outperform the state-of-the-art methods in terms of accuracy and efficiency.

Periocular region-based person identification in the visible, infrared and hyperspectral imagery

“…Experiments are repeated 10-fold by randomly selecting different gallery and probe combinations each time. For image set classification, six algorithms are used including AHISD, CHISD [18], SANP [22], DCC [19], MMD [20] and MDA [21]. For every algorithm, we performed 10-fold experiments for each of the four feature vector types.…”

“…Structural similarity of the sets is usually measured using subspace to subspace distance. Kim et al [19] proposed Discriminative Canonical Correlation (DCC) which performs discriminative learning on canonical correlations between the structures of image-sets. More specifically, a discriminant function is learned that maximized the within-class and minimized the between-class canonical correlations.…”

“…We extract four types of features from each periocular region and combine six state-of-the-art image-set classification techniques including the Affine Hull based image-set Distance (AHISD) [18], Convex Hull based image-set Distance (CHISD) [18], Discriminative Canonical Correlation (DCC) [19], Manifold-Manifold Distance (MMD) [20], Manifold Discriminant Analysis (MDA) [21], and Sparse Approximated Nearest Point (SANP) distance [22]. A brief overview of these techniques is given in Section 2.…”

HEp-2 fluorescence pattern classification

“…Leverage of large numbers of unlabelled HEp-2 images could allow development of much better understanding of the effect of imaging conditions on the resulting image texture, and compensating for these common variations in classification, for example through the use of manifold learning or subspace [16] methods.…”

Face Recognition and Retrieval in Video

“…Face registration methods can be adopted to deal with the effect of varying pose, for example, by utilizing the characteristic facial points (normally locations of the mouth and eyes). In some work (e.g., [59]), face recognition is performed directly on faces roughly localized, close to the conditions given by typical surveillance systems.…”