A depth-based nearest neighbor algorithm for high-dimensional data classification

Abstract

Nearest neighbor algorithms like k-nearest neighbors (kNN) are fundamental supervised learning techniques to classify a query instance based on class labels of its neighbors. However, quite often, huge volumes of datasets are not fully labeled and the unknown probability distribution of the instances may be uneven. Moreover, kNN suffers from challenges like curse of dimensionality, setting the optimal number of neighbors, and scalability for high-dimensional data. To overcome these challenges, we propose an improvised approach of classification via depth representation of subspace clusters formed from high-dimensional data. We offer a consistent and principled approach to dynamically choose the nearest neighbors for classification of a query point by i) identifying structures and distributions of data; ii) extracting relevant features, and iii) deriving an optimum value of k depending on the structure of data by representing data using data depth function. We propose an improvised classification algorithm using a depth-based representation of clusters, to improve performance in terms of execution time and accuracy. Experimentation on real-world datasets reveals that proposed approach is at least two orders of magnitude faster for high-dimensional dataset and is at least as accurate as traditional kNN.

İnşaat Ekibi Üretkenlik Tahmininde K-En Yakın Komşu ve Genelleştirilmiş Sinir Ağının Performansının Karşılaştırılması

“…While K-NN and GRNN are reported and used to be two different prediction methods, they are both based on Radial Basis Function Kernel (RBF Kernel) network and both do not require an iterative training procedure as back propagation networks. They approximate any arbitrary function between input and output data set, drawing the function estimate directly from the training data [13] Meanwhile the main difference between these two methods is that; while GRNN uses all of the training data sets for prediction of the output of a query, K-NN uses k number of data sets nearest to the query data set for prediction [14]. Like GRNN, K-NN also finds the distances between a query and all the examples in the training data set but unlike GRNN it only selects the specified number (k) of examples closest to the query and determines the similarities of only K-nearest neighbour to the query.…”

Improving collaborative recommendation based on item weight link prediction

“…Then it selects the active user's neighbourhood (top similar neighbours) to calculate the prediction value of an item. Finally, the algorithm compares the prediction value to a given threshold for the recommendation decision [17]. In order to reduce the computational complexity, model CF uses a part of the rating matrix to estimate or learn a model that generates the predictions.…”