Mirror Descent for Metric Learning: A Unified Approach

Kunapuli, Gautam; Shavlik, Jude W.

doi:10.1007/978-3-642-33460-3_60

Cited by 27 publications

(47 citation statements)

References 20 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bian et al [34] propose a constrained empirical risk minimization framework for distance metric learning (CDML). Kunapuli et al [35] propose an online regularized metric learning algorithm based on composite objective mirror descent (COMID). Huang et al [36] intend to propose a unified framework for sparse metric learning.…”

Section: Related Workmentioning

confidence: 99%

Online Similarity Learning for Big Data with Overfitting

Cong

Liu

Fan

et al. 2018

IEEE Trans. Big Data

View full text Add to dashboard Cite

Abstract-In this paper, we propose a general model to address the overfitting problem in online similarity learning for big data, which is generally generated by two kinds of redundancies: 1) feature redundancy, that is there exists redundant (irrelevant) features in the training data; 2) rank redundancy, that is non-redundant (or relevant) features lie in a low rank space. To overcome these, our model is designed to obtain a simple and robust metric matrix through detecting the redundant rows and columns in the metric matrix and constraining the remaining matrix to a low rank space. To reduce feature redundancy, we employ the group sparsity regularization, i.e., the ' 2;1 norm, to encourage a sparse feature set. To address rank redundancy, we adopt the low rank regularization, the max norm, instead of calculating the SVD as in traditional models using the nuclear norm. Therefore, our model can not only generate a low rank metric matrix to avoid overfitting, but also achieves feature selection simultaneously. For model optimization, an online algorithm based on the stochastic proximal method is derived to solve this problem efficiently with the complexity of Oðd 2 Þ. To validate the effectiveness and efficiency of our algorithms, we apply our model to online scene categorization and synthesized data and conduct experiments on various benchmark datasets with comparisons to several state-of-the-art methods. Our model is as efficient as the fastest online similarity learning model OASIS, while performing generally as well as the accurate model OMLLR. Moreover, our model can exclude irrelevant / redundant feature dimension simultaneously.

show abstract

Section: Related Workmentioning

confidence: 99%

Online Similarity Learning for Big Data with Overfitting

Cong

Liu

Fan

et al. 2018

IEEE Trans. Big Data

View full text Add to dashboard Cite

show abstract

“…In the case of PSD matrices, it is equivalent to the trace norm (also known as the nuclear norm) which favors low-rank solutions [see Kunapuli and Shavlik, 2012, for another approach based on the trace norm]. In the case of PSD matrices, it is equivalent to the trace norm (also known as the nuclear norm) which favors low-rank solutions [see Kunapuli and Shavlik, 2012, for another approach based on the trace norm].…”

Section: Mahalanobis Distance Learning 23mentioning

confidence: 99%

Metric Learning

Bellet

Habrard²,

Sebban³

2015

Synthesis Lectures on Artificial Intelligence and Machine Learn

105

103

View full text Add to dashboard Cite

International audienceSimilarity between objects plays an important role in both human cognitive processes and artificial systems for recognition and categorization. How to appropriately measure such similarities for a given task is crucial to the performance of many machine learning, pattern recognition and data mining methods. This book is devoted to metric learning, a set of techniques to automatically learn similarity and distance functions from data that has attracted a lot of interest in machine learning and related fields in the past ten years. In this book, we provide a thorough review of the metric learning literature that covers algorithms, theory and applications for both numerical and structured data. We first introduce relevant definitions and classic metric functions, as well as examples of their use in machine learning and data mining. We then review a wide range of metric learning algorithms, starting with the simple setting of linear distance and similarity learning. We show how one may scale-up these methods to very large amounts of training data. To go beyond the linear case, we discuss methods that learn nonlinear metrics or multiple linear metrics throughout the feature space, and review methods for more complex settings such as multi-task and semi-supervised learning. Although most of the existing work has focused on numerical data, we cover the literature on metric learning for structured data like strings, trees, graphs and time series. In the more technical part of the book, we present some recent statistical frameworks for analyzing the generalization performance in metric learning and derive results for some of the algorithms presented earlier. Finally, we illustrate the relevance of metric learning in real-world problems through a series of successful applications to computer vision, bioinformatics and information retrieval

show abstract

“…Then the metric learning model is specified as a constrained optimization problem. Quite a few online metric learning algorithms have been proposed, under an approach that learns a Mahalanobis matrix [10], [11], [12], [13]. In the sequel we describe some of the most well-known.…”

Section: Related Workmentioning

confidence: 99%

“…Unlike POLA, the authors perform the update solving a convex quadratic program [3]. In MDML (Mirror Descent Metric Learning) [12] the authors proposed a general framework for online Mahalanobis distance learning. It is based on composite mirror descent and is focused on regularization with the nuclear norm [3].…”

Section: A Online Mahalanobis Metric Learning Algorithmsmentioning

confidence: 99%

Mahalanobis distance metric learning algorithm for instance-based data stream classification

Pérez

Ribeiro

Morell

2016

2016 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Abstract-With the massive data challenges nowadays and the rapid growing of technology, stream mining has recently received considerable attention. To address the large number of scenarios in which this phenomenon manifests itself suitable tools are required in various research fields. Instance-based data stream algorithms generally employ the Euclidean distance for the classification task underlying this problem. A novel way to look into this issue is to take advantage of a more flexible metric due to the increased requirements imposed by the data stream scenario. In this paper we present a new algorithm that learns a Mahalanobis metric using similarity and dissimilarity constraints in an online manner. This approach hybridizes a Mahalanobis distance metric learning algorithm and a k-NN data stream classification algorithm with concept drift detection. First, some basic aspects of Mahalanobis distance metric learning are described taking into account key properties as well as online distance metric learning algorithms. Second, we implement specific evaluation methodologies and comparative metrics such as Q statistic for data stream classification algorithms. Finally, our algorithm is evaluated on different datasets by comparing its results with one of the best instance-based data stream classification algorithm of the state of the art. The results demonstrate that our proposal is better in some scenarios and has shown to be competitive in others.

show abstract

Mirror Descent for Metric Learning: A Unified Approach

Cited by 27 publications

References 20 publications

Online Similarity Learning for Big Data with Overfitting

Online Similarity Learning for Big Data with Overfitting

Metric Learning

Mahalanobis distance metric learning algorithm for instance-based data stream classification

Contact Info

Product

Resources

About