Learning Deep Match Kernels for Image-Set Classification

Sun, Haoliang; Zhen, Xiantong; Zheng, Yuanjie; Yang, Gongping; Yin, Yilong; Li, Shuo

doi:10.1109/cvpr.2017.661

Cited by 31 publications

(26 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The MDSD dataset [24] has been used for the task of dynamic scene classification. Following the settings in [27], we test the method based on the protocol of seventythirty-ratio (STR) which chooses seven videos for training and three videos for testing in each class.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

More About Covariance Descriptors for Image Set Coding: Log-Euclidean Framework Based Kernel Matrix Representation

Chen

Ren

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

We consider a family of structural descriptors for visual data, namely covariance descriptors (CovDs) that lie on a non-linear symmetric positive definite (SPD) manifold, a special type of Riemannian manifolds. We propose an improved version of CovDs for image set coding by extending the traditional CovDs from Euclidean space to the SPD manifold. Specifically, the manifold of SPD matrices is a complete inner product space with the operations of logarithmic multiplication and scalar logarithmic multiplication defined in the Log-Euclidean framework. In this framework, we characterise covariance structure in terms of the arc-cosine kernel which satisfies Mercer's condition and propose the operation of mean centralization on SPD matrices. Furthermore, we combine arc-cosine kernels of different orders using mixing parameters learnt by kernel alignment in a supervised manner. Our proposed framework provides a lower-dimensional and more discriminative data representation for the task of image set classification. The experimental results demonstrate its superior performance, measured in terms of recognition accuracy, as compared with the state-of-the-art methods.

show abstract

Section: Methodsmentioning

confidence: 99%

“…where W = 1 is a regularization term. β is defined by β i = T r(K i w K T ), whereK i w is the centralized matrix of the kernel matrix K i w [7,27], and the matrix Ω is defined by Ω i,j = T r(K i wK j w ). According to Proposition 2 in [7], the solution W * of Eq.20 is given by:…”

Section: Learning Weights Via Kernel Alignmentmentioning

confidence: 99%

More About Covariance Descriptors for Image Set Coding: Log-Euclidean Framework Based Kernel Matrix Representation

Chen

Ren

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

show abstract

“…(10) and Eq. (11) are no longer singular. For convenience, they are abbreviated by eig and exp in the following parts, respectively.…”

Section: Regularizationmentioning

confidence: 98%

Discriminative Residual Analysis for Image Set Classification With Posture and Age Variations

Ren

Luo

et al. 2020

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Image set recognition has been widely applied in many practical problems like real-time video retrieval and image caption tasks. Due to its superior performance, it has grown into a significant topic in recent years. However, images with complicated variations, e.g., postures and human ages, are difficult to address, as these variations are continuous and gradual with respect to image appearance. Consequently, the crucial point of image set recognition is to mine the intrinsic connection or structural information from the image batches with variations. In this work, a Discriminant Residual Analysis (DRA) method is proposed to improve the classification performance by discovering discriminant features in related and unrelated groups. Specifically, DRA attempts to obtain a powerful projection which casts the residual representations into a discriminant subspace. Such a projection subspace is expected to magnify the useful information of the input space as much as possible, then the relation between the training set and the test set described by the given metric or distance will be more precise in the discriminant subspace. We also propose a nonfeasance strategy by defining another approach to construct the unrelated groups, which help to reduce furthermore the cost of sampling errors. Two regularization approaches are used to deal with the probable small sample size problem. Extensive experiments are conducted on benchmark databases, and the results show superiority and efficiency of the new methods.

show abstract

“…• Relation with [49]: In fact, the proposed algorithm is an extension of our previous work [49]. The essential differences between the proposed work and the conference paper lie in the following five aspects: 1) in addition to set modeling with covariance matrix and linear subspace in [49], this paper also exploits gaussian distribution to encode the original set data for the sake of mining more useful information of intra-class variations; 2) Due to the space formed by a set of gaussian distributions is another Riemannian manifold Sym + d+1 , a well-equipped Riemannian kernel function is further applied to it for the purpose of preserving the structural information of the Riemannian manifold-valued data in the Hilbert space embedding; 3) Due to the discriminability of each local region in the produced kernel spaces is different, this paper integrates the devised multi-kernel learning algorithm into our originally proposed metric learning framework [49] for the sake of learning an adaptive weight for each, while [49] assigns the same weight to them; 4) To optimize the transformation matrix, this paper follows an efficient way [48] to directly solve the trace ratio problem, while [49] transforms this problem into a simpler ratio trace problem for approximation computing; 5) Besides the video-based face recognition and set-based object categorization tasks in the conference paper, we further assess the proposed work on video-based emotion recognition and dynamic scene classification tasks by making extensive experiments on two challenging videobased datasets: AFEW [50] and MDSD [37]. • Relation with [33]: The proposed method and [33] not only focus on building reliable set models, but also focus on learning discriminative subspace feature representations.…”

Section: F Relation With the Previous Workmentioning

confidence: 99%

“…Recently, image set classification has provided a new direction to address this task. In this paper, we report the classification performance of our method on the MDSD [37], [56] dataset. This dataset is comprised of 13 different categories of dynamic scenes with each has 10 video sequences.…”

Section: E Dynamic Scene Classificationmentioning

confidence: 99%

Multiple Riemannian Manifold-Valued Descriptors Based Image Set Classification With Multi-Kernel Metric Learning

Wang

Chen

et al. 2022

IEEE Trans. Big Data

View full text Add to dashboard Cite

The importance of wild video based image set recognition is becoming monotonically increasing due to the large amount of video resources obtained by diversified video collection approaches, like surveillance, drive recorders, smart phones, and internet. However, the contents of these collected videos are often complicated, and how to efficiently perform set modeling and feature extraction is a big challenge for set-based classification algorithms. In recent years, some proposed image set classification methods have made a considerable advance by modeling the original image set with covariance matrix, linear subspace, or Gaussian distribution. Moreover, the distinctive geometry spanned by them are Symmetric Positive Definite (SPD) manifold, Grassmann manifold, and Gaussian embedded Riemannian manifold, respectively. As a matter of fact, most of them just adopt a single geometric model to describe each given image set, which may lose some other useful information for classification. To tackle this problem, we propose a novel algorithm to model each image set from a multi-geometric perspective. Specifically, the covariance matrix, linear subspace, and Gaussian distribution are applied for set representation simultaneously. In order to fuse these multiple heterogeneous Riemannian manifoldvalued features, the well-equipped Riemannian kernel functions are first utilized to map them into high dimensional Hilbert spaces. Then, a multi-kernel metric learning framework is devised to embed the learned hybrid kernels into a lower dimensional common subspace for classification. We conduct experiments on four widely used datasets corresponding to four different classification tasks: video-based face recognition, set-based object categorization, video-based emotion recognition, and dynamic scene classification, to evaluate the classification performance of the proposed algorithm. Extensive experimental results justify its superiority over the state-of-the-art.

show abstract

Learning Deep Match Kernels for Image-Set Classification

Cited by 31 publications

References 36 publications

More About Covariance Descriptors for Image Set Coding: Log-Euclidean Framework Based Kernel Matrix Representation

More About Covariance Descriptors for Image Set Coding: Log-Euclidean Framework Based Kernel Matrix Representation

Discriminative Residual Analysis for Image Set Classification With Posture and Age Variations

Multiple Riemannian Manifold-Valued Descriptors Based Image Set Classification With Multi-Kernel Metric Learning

Contact Info

Product

Resources

About