Random forests for metric learning with implicit pairwise position dependence

Abstract

Metric learning makes it plausible to learn semantically meaningful distances for complex distributions of data using label or pairwise constraint information. However, to date, most metric learning methods are based on a single Mahalanobis metric, which cannot handle heterogeneous data well. Those that learn multiple metrics throughout the feature space have demonstrated superior accuracy, but at a severe cost to computational efficiency. Here, we adopt a new angle on the metric learning problem and learn a single metric that is able to implicitly adapt its distance function throughout the feature space. This metric adaptation is accomplished by using a random forest-based classifier to underpin the distance function and incorporate both absolute pairwise position and standard relative position into the representation. We have implemented and tested our method against state of the art global and multi-metric methods on a variety of data sets. Overall, the proposed method outperforms both types of method in terms of accuracy (consistently ranked first) and is an order of magnitude faster than state of the art multi-metric methods (16x faster in the worst case).

Context-Aware Hypergraph Modeling for Re-identification and Summarization

“…To overcome this, several multi-metric techniques have been proposed [37], [38] and they are often affected by the memory storage complexity since the matrixes need to be stored per point or the subset of points. In contrast, Random Forest Distance function [39] provides a non-parametric and non-linear distance metric that achieves the efficiency of both global and multi-metric techniques.…”

My Text in Your Handwriting

“…This allows us to learn the cost, using the two ligature examples in the training set, and transfer it to the missing ligature cases. The distance learning technique of Xiong et al [2012] is used; it makes use of regression forests [Breiman 2001;Criminisi and Shotton 2013]. Feature vectors are defined for every pair of adjacent glyphs, regardless of whether they have ligatures.…”

Target Detection Based on Random Forest Metric Learning

“…The above single-metric learning methods have been proved to be efficient, but they have a weak ability to deal with complex data, and they also have some other problems [27]. For example, the LMNN method has difficulties in dealing with high-dimensional data, and also requires a number of parameters to be adjusted.…”

“…Meanwhile, in the last few years, machine learning theory has been introduced into the HSI processing field, including classification [23]- [27]. To date, only a few researchers have proposed the use of distance metric learning in the machine learning domain for target detection, although lots of algorithms have shown good performance in classification applications based on distance metric learning methods [24], [25].…”