A k nearest neighbours classifiers ensemble based on extended neighbourhood rule and features subsets

Abstract

kNN based ensemble methods minimise the effect of outliers by identifying a set of data points in the given feature space that are nearest to an unseen observation in order to predict its response by using majority voting. The ordinary ensembles based on kNN find out the k nearest observations in a region (bounded by a sphere) based on a predefined value of k. This scenario, however, might not work in situations when the test observation follows the pattern of the closest data points with the same class that lie on a certain path not contained in the given sphere. This paper proposes a k nearest neighbour ensemble where the neighbours are determined in k steps. Starting from the first nearest observation of the test point, the algorithm identifies a single observation that is closest to the observation at the previous step. At each base learner in the ensemble, this search is extended to k steps on a random bootstrap sample with a random subset of features selected from the feature space. The final predicted class of the test point is determined by using a majority vote in the predicted classes given by all base models. This new ensemble method is applied on 17 benchmark datasets and compared with other classical methods, including kNN based models, in terms of classification accuracy, kappa and Brier score as performance metrics. Boxplots are also utilised to illustrate the difference in the results given by the proposed and other state-of-the-art methods. The proposed method outperformed the rest of the classical methods in the majority of cases. The paper gives a detailed simulation study for further assessment.

Radar-optical fusion detection of UAV based on improved YOLOv7-tiny

“…Step 3: Calculation of the geometric form centre deviation. First, the geometric centre of the M camera target topologies was obtained [28,29], denoted as C c = (C u , C v ). Subsequently, find the geometric topocentre of the set of M new radar targets R ′ , denoted as…”

A k-Nearest Neighbours Based Ensemble via Optimal Model Selection for Regression

“…This paper introduces a novel ensemble method called random projection extended neighbourhood rule (RPExNRule) for kNN. This procedure constructs a large number of the extended neighbourhood rule (ExNRule) based classifiers [24], each on a randomly projected bootstrap sample and a new data point is predicted by all these models. The final prediction class is obtained using majority voting among the results given by the underlying models.…”

“…χ 0 1×p ′ . Then apply the extended neighbourhood rule (ExNRule) [24] to each random projection and estimate the test data point. The predictions given by all B models of the unseen observation X 1×p are Ŷ 1 , Ŷ 2 , Ŷ 3 , .…”

“…The proposed method takes a large number of bootstrap samples and the feature space of each sample is randomly projected. A model based on ExNRule [24] is fitted to each of these projections.…”

“…A novel random projection extended neighbourhood rule (RPExNRule) for the kNN ensemble is proposed in this paper. The extended neighbourhood rule (ExNRule) [24] is used for kNN as base learners. To construct each base model a bootstrap sample is drawn from the training data with the full feature space and then randomly projected into a reduced dimension.…”