Combining Multiple Kernels by Augmenting the Kernel Matrix

Yan, Fei; Mikolajczyk, Krystian; Kittler, Josef; Tahir, Muhammad Atif

doi:10.1007/978-3-642-12127-2_18

Cited by 11 publications

(16 citation statements)

References 17 publications

(21 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The primal and its corresponding dual for a linear combination of kernels are derived for various formulations in [1,9,10,19]. In contrast, in AKM [23], given a set of base training kernels (K p ) the augmented kernel is defined as follows:…”

Section: Akm and Its Correspondence To Classifier Fusionmentioning

confidence: 99%

“…The key idea of MKL, is to learn a linear combination of base kernels by maximizing soft margin between classes [10]. Alternatively, AKM [23] is proposed arguing that in MKL a single kernel corresponding to a particular feature space is attributed a single weight. Therefore, MKL does not exploit information from individual samples in different feature spaces, e.g., in the context of object recognition, some samples can carry more shape information while others may carry more texture information for the same object category.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Augmented Kernel Matrix vs Classifier Fusion for Object Recognition

Awais¹,

Yan²,

Mikolajczyk³

et al. 2011

Procedings of the British Machine Vision Conference 2011

Self Cite

View full text Add to dashboard Cite

Augmented Kernel Matrix (AKM) has recently been proposed to accommodate for the fact that a single training example may have different importance in different feature spaces, in contrast to Multiple Kernel Learning (MKL) that assigns the same weight to all examples in one feature space. However, the AKM approach is limited to small datasets due to its memory requirements. An alternative way to fuse information from different feature channels is classifier fusion (ensemble methods). There is a significant amount of work on linear programming formulations of classifier fusion (CF) in the case of binary classification. In this paper we derive primal and dual of AKM to draw its correspondence with CF. We propose a multiclass extension of binary ν-LPBoost, which learns the contribution of each class in each feature channel. Existing approaches of CF promote sparse features combinations, due to regularization based on 1 -norm, and lead to a selection of a subset of feature channels, which is not good in case of informative channels. We also generalize existing CF formulations to arbitrary p -norm for binary and multiclass problems which results in more effective use of complementary information. We carry out an extensive comparison and show that the proposed nonlinear CF schemes outperform its sparse counterpart as well as state-of-the-art MKL approaches.

show abstract

Section: Akm and Its Correspondence To Classifier Fusionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Augmented Kernel Matrix vs Classifier Fusion for Object Recognition

Awais¹,

Yan²,

Mikolajczyk³

et al. 2011

Procedings of the British Machine Vision Conference 2011

Self Cite

View full text Add to dashboard Cite

show abstract

“…In essence, MKL is simply a linear combination of different kernels. It implies that the contribution of each kernel is fixed for all the training samples [19]. This seems to be an unnecessary too strong constraint.…”

Section: Related Workmentioning

confidence: 99%

“…This seems to be an unnecessary too strong constraint. In [19], the authors propose to learn augmented coeffi-cients for each sample in each feature channel. They achieve this by augmenting the kernel matrixes.…”

Section: Related Workmentioning

confidence: 99%

Feature Combination beyond Basic Arithmetics

Fu¹,

Qiu²,

He³

2011

Procedings of the British Machine Vision Conference 2011

View full text Add to dashboard Cite

Kernel-based feature combination techniques such as Multiple Kernel Learning use arithmetical operations to linearly combine different kernels. We have observed that the kernel distributions of different features are usually very different. We argue that the similarity distributions amongst the data points for a given dataset should not change with their representation features and propose the concept of relative kernel distribution invariance (RKDI). We have developed a very simple histogram matching based technique to achieve RKDI by transforming the kernels to a canonical distribution. We have performed extensive experiments on various computer vision and machine learning datasets and show that calibrating the kernels to an empirically chosen canonical space before they are combined can always achieve a performance gain over state-of-art methods. As histogram matching is a remarkably simple and robust technique, the new method is universally applicable to kernel-based feature combination.

show abstract

“…To deal with this issue, this paper makes use of the notion of empirical feature space [5,6], which preserves the geometrical structure of the original feature space, given that distances and angles in the feature space are uniquely determined by dot products and that the dot products of the corresponding images are the original kernel values. This empirical feature space is Euclidean, so it provides a tractable framework to study the spatial distribution of the mapping function Φ(·) [7], to measure class separability [6] and to optimize the kernel [6,8]. Besides, the notion of empirical kernel feature space has been used for the kernelization of all kinds of linear classifiers [9,10], with the advantage that the algorithm does not need to be formulated to deal with dot products.…”

Section: Introductionmentioning

confidence: 99%

Borderline Kernel Based Over-Sampling

Pérez-Ortiz

Gutiérrez

Hervás-Martı́nez

2013

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Nowadays, the imbalanced nature of some real-world data is receiving a lot of attention from the pattern recognition and machine learning communities in both theoretical and practical aspects, giving rise to different promising approaches to handling it. However, preprocessing methods operate in the original input space, presenting distortions when combined with kernel classifiers, that operate in the feature space induced by a kernel function. This paper explores the notion of empirical feature space (a Euclidean space which is isomorphic to the feature space and therefore preserves its structure) to derive a kernel-based synthetic over-sampling technique based on borderline instances which are considered as crucial for establishing the decision boundary. Therefore, the proposed methodology would maintain the main properties of the kernel mapping while reinforcing the decision boundaries induced by a kernel machine. The results show that the proposed method achieves better results than the same borderline over-sampling method applied in the original input space.

show abstract

Combining Multiple Kernels by Augmenting the Kernel Matrix

Cited by 11 publications

References 17 publications

Augmented Kernel Matrix vs Classifier Fusion for Object Recognition

Augmented Kernel Matrix vs Classifier Fusion for Object Recognition

Feature Combination beyond Basic Arithmetics

Borderline Kernel Based Over-Sampling

Contact Info

Product

Resources

About