Density Weighted K-Nearest Neighbors Algorithm for Outliers in the Training Set Are So Close to the Test Element

Abstract

Abstract:In KNN (K-Nearest Neighbour) method, the distance-weighted algorithm is applied in order to reduce the effect of noisy data. However, as this weighting algorithm will be insufficient when the outliers in the training set are so close to the test element, a new weighting algorithm is required for KNN. In this study, instead of distance-weighting, a new density-weighted KNN algorithm is proposed for reducing the effect of noisy data. In the first stage of the proposed method, the coefficient of density of each element in the training set was obtained by Parzen window method. And then, the membership of each test element was determined according to the total of density coefficients (weights) of neighbours belonging to the same class. As for the last stage, the performance results of the frequently used KNN methods and the proposed method (Density-weighted KNN, Classical KNN and Distance-weighted KNN) were compared. The obtained results have shown that the proposed method is more successful by almost 1% than classical KNN method, by 9% than distance weighted KNN method. Moreover, it is more successful than other KNN methods when the test element is so close to the training set elements.