Semi-supervised metric learning in stratified spaces via intergrating local constraints and information-theoretic non-local constraints

Karimi, Zohre; Ghidary, Saeed Shiry

doi:10.1016/j.neucom.2018.05.089

Cited by 9 publications

(8 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The machine learning literature studies on 𝑘NN classifier confirm that the accuracy of this classifier is highly dependent on the underlying metric specifying the nearest neighbors [6,11]. Learning a suitably parametrized distance function from data improves the generalization ability of kNN classifier.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…However, the underlying distance metric used significantly affects the performance of this classifier. While 𝑘NN takes advantage of Euclidean distance metric, significant studies demonstrate that using a fixed distance function to measure the distance between two objects is unsuitable, especially in the high-dimensional space [11,12]. The p-norm distance metric is a generalization of Euclidean distance, and several experiments confirm that the value of p affects the performance of 𝑘NN.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Adaptive k-nearest neighbor Classifier using Differential Evolution with Auto-Enhanced Population Diversity for Intrusion Detection

Karimi

Torabi

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Machine learning methods have attracted increasing interest in recent studies on intrusion detection. A classifier is applied to discriminate attacks from normal connections in these methods. 𝒌-nearest neighbor (𝒌NN) has been widely used in intrusion detection due to its simplicity and effectiveness. The classical 𝒌NN exploits Euclidean distance for identifying nearest neighbors, whereas how to compute the distance of data points is highly application-specific and plays a crucial role in the effectiveness of this classifier. In this paper, a novel 𝒌NN classifier is proposed that employs p-norm distance metric, the generalization of Euclidean distance, by learning p from data. The value of p in the proposed data-dependent metric is learned by the differential evolution algorithm exploiting auto-enhanced population diversity. The experimental results showed significant improvements in terms of F1 score and error rate compared to conventional kNN and Naive Bayesian classifiers on Kyoto2006+ and NSL-KDD. Furthermore, they verify the superiority of kNN classifier using the proposed data-dependent metric in terms of receiver operating characteristic curve and the corresponding area under the curve.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Adaptive k-nearest neighbor Classifier using Differential Evolution with Auto-Enhanced Population Diversity for Intrusion Detection

Karimi

Torabi

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Sometimes, metric learning is formulated as a projection of the data into a new feature space. Supervised DR related to metric learning aims at locating a low-dimensional representation that maximizes the separation of labeled data, such as neighborhood component analysis (NCA) [15], maximally collapsing metric learning (MCML) [14], LMNN [16], global and local metric learning [17], and information-theoretic metric learning [18]. Specifically, LMNN is designed to learn a Mahalanobis distance metric for the nearest neighbor graph (KNN) classification.…”

Section: Related Workmentioning

confidence: 99%

Efficient Linear Feature Extraction Based on Large Margin Nearest Neighbor

Zhao

Zhou

2019

IEEE Access

View full text Add to dashboard Cite

Linear feature extraction methods have become indispensable tools in pattern recognition. The linear dimensionality reduction optimizes some objective to produce a linear transformation and derives the discriminative low-dimensional transformed data wherein the similarly labeled samples cluster tightly and the differently labeled samples keep away from one another. In the past, most of the methods achieve between-class distance by maximizing between-class-center-mean to make the differently labeled samples separated from each other. However, the samples in this study located in the class margin or within the other classes called hard samples often move slowly, which results in a small between-class distance and deteriorates the classification performance. Hence, we utilize the large margin nearest neighbor (LMNN) to quickly push only the hard samples toward the center of the class to get the large class margin instead of between-class, further improving the performance of classification. Combined with the linear discriminant analysis (LDA), a novel linear feature extraction method called LDA-LMNN is proposed. The method can address the limitations of LDA. Furthermore, the dropout is employed to the learning of linear transform matrix to improve the generalization ability and overcome the overfitting. The comparative experiments on various real-world datasets by using the state-of-the-art feature extraction methods demonstrate the effectiveness of the proposed method.INDEX TERMS Linear dimensionality reduction, large margin nearest neighbor, linear discriminant analysis, hard samples.

show abstract

“…The second category contains some weakly supervised methods that only require pairwise constraints that reflect similarity judgements between data pairs. Information-theoretic metric learning (ITML) [14][15] is a representative method proposed to learn one distance metric through entropy theory and the Bregman optimization problem. Relevant component analysis (RCA) [16][17] is another weakly supervised method that learns a global linear transformation by exploiting only relevant constraints.…”

Section: Introductionmentioning

confidence: 99%

A Novel Method of Efficient Max-min Metric for Classification

Liu

2023

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Distance metric learning is an important method to study distance metrics that reflect the interaction between features and labels. Because of the high computational complexity and the fact that existing studies on algorithms that measure the similarities with Euclidean distances cannot reflect the real correlations between pairs of samples, learning a suitable distance metric is quite demanding for many data mining tasks. This paper innovatively proposes an extended efficient max-min metric (EMM) that maximizes the total distance between different pairs and minimizes the total distance between similar pairs as much as possible. Simultaneously, the adoption of the local preserving projection framework changes the solution process of the algorithm and improves the speed of the algorithm without losing accuracy. Because traditional EMM only considers pairwise constraints and ignores sample distribution, this study extends EMM based on sample distribution and successfully solves the multi-manifold problem. In the process of data realization, compared with the vector representation method, the use of high-order tensors will make the image representation more accurate and natural. To maintain the structure of higher-order tensors, a tensor-efficient max-min metric (TEMM) is proposed. In order to prove the accuracy and superiority of the research method in this paper, a large number of experiments have been carried out on image processing. The experimental results show that the method proposed in this paper has a good effect.

show abstract

Semi-supervised metric learning in stratified spaces via intergrating local constraints and information-theoretic non-local constraints

Cited by 9 publications

References 36 publications

An Adaptive k-nearest neighbor Classifier using Differential Evolution with Auto-Enhanced Population Diversity for Intrusion Detection

An Adaptive k-nearest neighbor Classifier using Differential Evolution with Auto-Enhanced Population Diversity for Intrusion Detection

Efficient Linear Feature Extraction Based on Large Margin Nearest Neighbor

A Novel Method of Efficient Max-min Metric for Classification

Contact Info

Product

Resources

About