Mixtures of Large Margin Nearest Neighbor Classifiers

Abstract

Abstract. The accuracy of the k-nearest neighbor algorithm depends on the distance function used to measure similarity between instances. Methods have been proposed in the literature to learn a good distance function from a labelled training set. One such method is the large margin nearest neighbor classifier that learns a global Mahalanobis distance. We propose a mixture of such classifiers where a gating function divides the input space into regions and a separate distance function is learned in each region in a lower dimensional manifold. We show that such an extension improves accuracy and allows visualization.

Multiple Cayley-Klein metric learning

“…Since our method learns several Cayley-Klein metrics locally and combines them together for a global and powerful distance metric, it is mostly related to the local metric learning [ 11 , 13 , 15 , 32 ] and some mixed/compositional metric learning methods [ 16 , 33 ]. MM-LMNN [ 11 ] is an extension of LMNN which learns a small number of metrics (typically one per class) in an effort to alleviate overfitting.…”

“…Additionally, a nuclear norm regularizer is adopted to obtain low-rank weight matrices for calculating metrics, which is able to control the rank of the involved linear mappings through a sparsity-inducing matrix norm. Recently, Semerci and Alpaydin [ 16 ] proposed the Mixture of LMNN (MoLMNN) method to learn a mixture of local Mahalanobis distances to better discriminate the data. It needs a gating function to softly partition the input space into several regions.…”

“…Therefore, researchers have resorted to more complicated non-linear metrics to pursue a higher performance. These attempts include local metric learning [ 11 , 13 – 16 ], kernel metric learning [ 17 ] and the most recently proposed Cayley-Klein metric learning [ 18 ], etc.…”

Modeling Perceptual Color Differences by Local Metric Learning

“…In [26], Wang et al optimize a combination of metric bases that are learned for some anchor points defined as the means of clusters constructed, for example, by the K-Means algorithm. Other local metric learning algorithms have been recently proposed, only in a classification setting, such as [33] which makes use of random forests and absolute position of points to compute a local metric; in [10], a local metric is learned based on a conical combination of Mahalanobis metrics and pair-wise similarities between the data; a last example of this non exhaustive list comes from [21], where the authors learn a mixture of local Mahalanobis distances.…”