Prototype and Feature Selection by Sampling and Random Mutation Hill Climbing Algorithms

Abstract

With the goal of reducing computational costs without sacrificing accuracy, we describe two algorithms to find sets of prototypes for nearest neighbor classification. Here, the term "prototypes" refers to the reference instances used in a nearest neighbor computation-the instances with respect to which similarity is assessed in order to assign a class to a new data item. Both algorithms rely on stochastic techniques to search the space of sets of prototypes and are simple to implement. The first is a Monte Carlo sampling algorithm; the second applies random mutation hill climbing. On four datasets we show that only three or four prototypes sufficed to give predictive accuracy equal or superior to a basic nearest neighbor algorithm whose run-time storage costs were approximately 10 to 200 times greater. We briefly investigate how random mutation hill climbing may be applied to select features and prototypes simultaneously. Finally, we explain the performance of the sampling algorithm on these datasets in terms of a statistical measure of the extent of clustering displayed by the target classes.

Comparison of Instances Seletion Algorithms I. Algorithms Survey

“…These two methods described in [11] by Skalak are based on stochastic behavior. MC1 in each iteration use Monte Carlo to draw new set of instances and to test the accuracy.…”

“…A group of three algorithms were inspired by encoding length principle [8]. Other algorithms were derived from graph theory [9], sets theory [10] or Monte Carlo sampling [11].…”

Supervised Feature Ranking Using a Genetic Algorithm Optimized Artificial Neural Network

“…5 GA based frature selection methods 6 are usually found to perform better than other heuristic search methods for large sized data sets; however, they also require considerable computation time for large data sets. Other attempts to decrease the computational time of feature selection include probabilistic search methods such as random hill climbing, 7 SCHEMATA+, 8 and Las Vegas Filter approach. 9 The artificial neural networks (ANNs) have been grown in popularity and used extensively to perform various classification tasks.…”

A new maximum relevance-minimum multicollinearity (MRmMC) method for feature selection and ranking

“…Thus, it is often preferable for many high dimensional problems to employ heuristic methods that compromise subset optimality for better computational efficiency. A few examples of such search strategies are sequential search [5,6], floating search [7][8][9], random mutation hill climbing [10] and evolutionary-based approaches [11][12][13][14] .…”