A Hybrid SVM Based on Nearest Neighbor Rule

Abstract

This paper proposes a hybrid learning method to speed up the classification procedure of Support Vector Machines (SVM). Comparing most algorithms trying to decrease the support vectors in an SVM classifier, we focus on reducing the data points that need SVM for classification, and reduce the number of support vectors for each SVM classification. The system uses a Nearest Neighbor Classifier (NNC) to treat data points attentively. In the training phase, the NNC selects data near partial decision boundary, and then trains sub SVM for each Voronoi pair. For classification, most non-boundary data points are classified by NNC directly, while remaining boundary data points are passed to a corresponding local expert SVM. We also propose a data selection method for training reliable expert SVM. Experimental results on several generated and public machine learning data sets show that the proposed method significantly accelerates the testing speed.

Nearest neighbor ensembles for functional data with interpretable feature selection

“…For example, Gertheiss and Tutz [13] extended it to an ensemble of weighted nearest neighbor posterior probability estimates. Further extensions can be found in Ji and Zhao [14] and Hayat et al [15]; see also Bischl et al [16] for a comparison of the k-nearest-neighbor approach with other local discrimination techniques.…”