Evaluation of k-nearest neighbour classifier performance for heterogeneous data sets

Abstract

Distance-based algorithms are widely used for data classification problems. The k-nearest neighbour classification (k-NN) is one of the most popular distance-based algorithms. This classification is based on measuring the distances between the test sample and the training samples to determine the final classification output. The traditional k-NN classifier works naturally with numerical data. The main objective of this paper is to investigate the performance of k-NN on heterogeneous datasets, where data can be described as a mixture of numerical and categorical features. For the sake of simplicity, this work considers only one type of categorical data, which is binary data. In this paper, several similarity measures have been defined based on a combination between well-known distances for both numerical and binary data, and to investigate k-NN performances for classifying such heterogeneous data sets. The experiments used six heterogeneous datasets from different domains and two categories of measures. Experimental results showed that the proposed measures performed better for heterogeneous data than Euclidean distance, and that the challenges raised by the nature of heterogeneous data need personalised similarity measures adapted to the data characteristics.

An Experimental Analysis of Attack Classification Using Machine Learning in IoT Networks

“…KNN is a supervised learning model that is considered to be one of the simplest ML models available [ 13 ]. KNN is referred to as a lazy learner because there is no training done with KNN; instead, the training data are used when making predictions to classify the data [ 13 ]. KNN operates under the assumption that similar data points will group and finds the closest data points using the K value, which can be set to any number [ 14 ].…”

“…The second concept involves only using small subset of the features when splitting the nodes in the trees [ 23 ]. This is done to prevent overfitting when the model uses the training data to inflate the predictions made by the model [ 13 ]. When making predictions with RF, the average of each of the trees predictions is used to determine the overall class of the data; this process is called bootstrap aggregating [ 13 ].…”

The Application of Molecular Spectroscopy in Combination with Chemometrics for Halal Authentication Analysis: A Review

“…KNN is one of the most popular classification techniques based on distance algorithms. KNN is based on measuring the distances between the training samples and test samples to determine the final classification output [ 36 ].…”

Classification techniques’ performance evaluation for facial expression recognition

“…K-Nearest Neighbour classifier is designed to function based on measuring the distance. It attempts to classify numeral data records after finding the K-Nearest neighbor via measuring the distance between the training samples and the test samples according to Euclidian [33]. In this approach, the output is class-labled and K is usually a positive and integer digit of its neighbors.…”