Learning Local Metrics and Influential Regions for Classification

Dong, Mingzhi; Wang, Yujiang; Yang, Xiaochen; Xue, Jing-Hao

doi:10.1109/tpami.2019.2914899

Cited by 20 publications

(18 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The above setup resulted in problem sizes from 29 to 400. We applied the following two data normalization schemes for the training/test data: i) a standardization scheme in [41] that first subtracts the mean and divides by the feature-wise standard deviation, and then normalizes to unit length sample-wise, and ii) a min-max scheme [40] that rescales each feature to within 0 and 1. We added 10 −12 noise to the dataset to avoid NaN's due to data normalization on small samples.…”

Section: Methodsmentioning

confidence: 99%

Projection-free Graph-based Classifier Learning using Gershgorin Disc Perfect Alignment

Yang

Cheung

Tan

et al. 2021

Preprint

View full text Add to dashboard Cite

In semi-supervised graph-based binary classifier learning, a subset of known labels xi are used to infer unknown labels, assuming that the label signal x is smooth with respect to a similarity graph specified by a Laplacian matrix. When restricting labels x i to binary values, the problem is NP-hard. While a conventional semidefinite programming (SDP) relaxation can be solved in polynomial time using, for example, the alternating direction method of multipliers (ADMM), the complexity of iteratively projecting a candidate matrix M onto the positive semi-definite (PSD) cone (M 0) remains high. In this paper, leveraging a recent linear algebraic theory called Gershgorin disc perfect alignment (GDPA), we propose a fast projection-free method by solving a sequence of linear programs (LP) instead. Specifically, we first recast the SDP relaxation to its SDP dual, where a feasible solution H 0 can be interpreted as a Laplacian matrix corresponding to a balanced signed graph sans the last node. To achieve graph balance, we split the last node into two that respectively contain the original positive and negative edges, resulting in a new Laplacian H. We repose the SDP dual for solution H, then replace the PSD cone constraint H 0 with linear constraints derived from GDPAsufficient conditions to ensure H is PSD-so that the optimization becomes an LP per iteration. Finally, we extract predicted labels from our converged LP solution H. Experiments show that our algorithm enjoyed a 40× speedup on average over the next fastest scheme while retaining comparable label prediction performance.Preprint. Under review.

show abstract

Section: Methodsmentioning

confidence: 99%

Projection-free Graph-based Classifier Learning using Gershgorin Disc Perfect Alignment

Yang

Cheung

Tan

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Definition 2: [37] Let (X , d X ) and (Y, d Y ) be two metric spaces, the Lipschitz constant of a function f is defined as:…”

Section: Theoretical Analysis On the Inseparable Problem Of Metric Le...mentioning

confidence: 99%

“…where d is the rank of L [37]. Since 1 sm does not change the learning ability of f θ (x), the Lipschitz constant of linear metric learning has a very small upperbound.…”

Section: Theoretical Analysis On the Inseparable Problem Of Metric Le...mentioning

confidence: 99%

Adaptive neighborhood Metric learning

Song,

Han,

Cheng

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we reveal that metric learning would suffer from serious inseparable problem if without informative sample mining. Since the inseparable samples are often mixed with hard samples, current informative sample mining strategies used to deal with inseparable problem may bring up some side-effects, such as instability of objective function, etc. To alleviate this problem, we propose a novel distance metric learning algorithm, named adaptive neighborhood metric learning (ANML). In ANML, we design two thresholds to adaptively identify the inseparable similar and dissimilar samples in the training procedure, thus inseparable sample removing and metric parameter learning are implemented in the same procedure. Due to the non-continuity of the proposed ANML, we develop an ingenious function, named log-exp mean function to construct a continuous formulation to surrogate it, which can be efficiently solved by the gradient descent method. Similar to Triplet loss, ANML can be used to learn both the linear and deep embeddings. By analyzing the proposed method, we find it has some interesting properties. For example, when ANML is used to learn the linear embedding, current famous metric learning algorithms such as the large margin nearest neighbor (LMNN) and neighbourhood components analysis (NCA) are the special cases of the proposed ANML by setting the parameters different values. When it is used to learn deep features, the state-of-the-art deep metric learning algorithms such as Triplet loss, Lifted structure loss, and Multi-similarity loss become the special cases of ANML. Furthermore, the log-exp mean function proposed in our method gives a new perspective to review the deep metric learning methods such as Prox-NCA and N-pairs loss. At last, promising experimental results demonstrate the effectiveness of the proposed method.

show abstract

“…In this section, we introduce a new type of decision function of K-NN classifier proposed recently by [4]. This new type of K-NN classifier is started from the two-class case, i.e., to judge whether a sample belongs to a specially-pointed class.…”

Section: A Decision Function Of K-nn Classifiermentioning

confidence: 99%

“…For example, the cross-entropy loss widely used in deep learning classifiers comes from the traditional softmax regression [1], [2], [3]. The other side is that traditional classifiers are still valuable on the task with small data set [4], [5].…”

Section: Introductionmentioning

confidence: 99%

Adaptive Nearest Neighbor: A General Framework for Distance Metric Learning

Song¹

2019

Preprint

View full text Add to dashboard Cite

K-NN classifier is one of the most famous classification algorithms, whose performance is crucially dependent on the distance metric. When we consider the distance metric as a parameter of K-NN, learning an appropriate distance metric for K-NN can be seen as minimizing the empirical risk of K-NN. In this paper, we design a new type of continuous decision function of the K-NN classification rule which can be used to construct the continuous empirical risk function of K-NN. By minimizing this continuous empirical risk function, we obtain a novel distance metric learning algorithm named as adaptive nearest neighbor (ANN). We have proved that the current algorithms such as the large margin nearest neighbor (LMNN), neighbourhood components analysis (NCA) and the pairwise constraint methods are special cases of the proposed ANN by setting the parameter different values. Compared with the LMNN, NCA, and pairwise constraint methods, our method has a broader searching space which may contain better solutions. At last, extensive experiments on various data sets are conducted to demonstrate the effectiveness and efficiency of the proposed method.

show abstract

Learning Local Metrics and Influential Regions for Classification

Cited by 20 publications

References 22 publications

Projection-free Graph-based Classifier Learning using Gershgorin Disc Perfect Alignment

Projection-free Graph-based Classifier Learning using Gershgorin Disc Perfect Alignment

Adaptive neighborhood Metric learning

Adaptive Nearest Neighbor: A General Framework for Distance Metric Learning

Contact Info

Product

Resources

About