Natural Encoding for Evolutionary Supervised Learning

Abstract

Some of the most influential factors in the quality of the solutions found by an evolutionary algorithm (EA) are a correct coding of the search space and an appropriate evaluation function of the potential solutions. EAs are often used to learn decision rules from datasets, which are encoded as individuals in the genetic population. In this paper, the coding of the search space for the obtaining of those decision rules is approached, i.e., the representation of the individuals of the genetic population and also the design of specific genetic operators. Our approach, called "natural coding," uses one gene per feature in the dataset (continuous or discrete). The examples from the datasets are also encoded into the search space, where the genetic population evolves, and therefore the evaluation process is improved substantially. Genetic operators for the natural coding are formally defined as algebraic expressions. Experiments with several datasets from the University of California at Irvine (UCI) machine learning repository show that as the genetic operators are better guided through the search space, the number of rules decreases considerably while maintaining the accuracy, similar to that of hybrid coding, which joins the well-known binary and real representations to encode discrete and continuous attributes, respectively. The computational cost associated with the natural coding is also reduced with regard to the hybrid representation. Our algorithm, HIDER*, has been statistically tested against C4.5 and C4.5 Rules, and performed well. The knowledge models obtained are simpler, with very few decision rules, and therefore easier to understand, which is an advantage in many domains. The experiments with high-dimensional datasets showed the same good behavior, maintaining the quality of the knowledge model with respect to prediction accuracy. Index Terms-Decision rules, evolutionary encoding, supervised learning. I. INTRODUCTION D ECISION RULES are especially relevant in problems related to supervised learning. Given a dataset with continuous and discrete features or attributes, and a class label, we try to find a rule set that describes the knowledge within data or classifies new unseen data. When the feature is discrete, the rules take the form of "if , then class," where the values are not necessarily all those that the feature can take. When the feature is continuous, typically the rules take the form of "if then class," where

Opposition-Based Discrete PSO Using Natural Encoding for Classification Rule Discovery

“…The performance statistics of ONDPSO in terms of average percentage error, number of rules R, terms to rule ratio T/R and number of fitness evaluations FE are reported in Table 3. Table 4 shows the results where, for the seven datasets, the obtained percentage median accuracy achieved by ONDPSO has been compared with the results reported in [54]. As shown in Table 4, our proposed algorithm Figure 10 clearly shows that, most of the time during the testing phase, the algorithm stayed closer to its best performance -i.e., a low error rate.…”

Parallel evaluation of Pittsburgh rule-based classifiers on GPUs

“…The rule base is formed by the best rules obtained when the algorithm is run multiple times. SLAVE [37], SIA [38] and HIDER [39] are examples which follow this model.…”

An algorithm evaluation for discovering classification rules with gene expression programming

“…HIDER 34 : genetic rule-based algorithm that represents knowledge in a similar fashion to GASSIST as a hierarchical set of rules which take the form of a decision list. The main difference of HIDER is that it uses natural coding 35 to represent each rule. It uses an iterative rule learning approach.…”