Improving k-Nearest Neighbour Classification with Distance Functions Based on Receiver Operating Characteristics

Abstract

Abstract. The k-nearest neighbour (k-NN) technique, due to its interpretable nature, is a simple and very intuitively appealing method to address classification problems. However, choosing an appropriate distance function for k-NN can be challenging and an inferior choice can make the classifier highly vulnerable to noise in the data. In this paper, we propose a new method for determining a good distance function for k-NN. Our method is based on consideration of the area under the Receiver Operating Characteristics (ROC) curve, which is a well known method to measure the quality of binary classifiers. It computes weights for the distance function, based on ROC properties within an appropriate neighbourhood for the instances whose distance is being computed. We experimentally compare the effect of our scheme with a number of other well-known k-NN distance metrics, as well as with a range of different classifiers. Experiments show that our method can substantially boost the classification performance of the k-NN algorithm. Furthermore, in a number of cases our technique is even able to deliver better accuracy than state-of-the-art non k-NN classifiers, such as support vector machines.

Incorporating feature ranking and evolutionary methods for the classification of high-dimensional DNA microarray gene expression data

“…For the next step, we compared our models as well as the baseline XCS with several other machine learning methods, either with a feature selection method in place or without using any feature selection. The results are presented in Table 3, which are cross-checked with the results published by Hossain et al 1 and Hassan et al 2 From Figures 3 and 4, we can claim that on average Information Gain delivers better results than other feature ranking methods for our models. Thus, we have used information Gain as the feature selection approach in conjunction with FS-XCS, GRD-XCS and other machine learning methods.…”

An Automated CAD System of CT Chest Images for COVID-19 Based on Genetic Algorithm and K-Nearest Neighbor Classifier

“…In this paper, the selected features are applied to classification algorithms such as KNN [37] and decision tree [38]. The two classifiers are trained on the selected features based on binary classifiers by labeling zero as non-coronavirus and one as coronavirus images.…”

A Neural Network Based Distance Function for the k-Nearest Neighbor Classifier

“…A similar attempt is proposed by Wang et al [7], where the authors proved that the a simple adaptive distance measure, which basically normalizes the ordinary Euclidean or Manhattan distance from an unknown pattern to each labeled training sample -by the shortest distance between the corresponding training sample to training samples belonging to different class -, can be considered the leading factor to improve upon the original kNN classifier. Hassan et al [1] proposed a weighting technique based on ROC (Receiver Operating Characteristic) curve.…”

“…These metrics due to their properties (symmetry, identity and triangle inequality) can be used on all kind of metric spaces, however this advantage is their disadvantage too. For these norms, the different feature dimensionalities are weighted equally [1], which can be considered a major disadvantage [2]. In many cases different features contribute differently in the feature space, thus being inadequate for proper classical distance calculus.…”