A genetic-algorithm for discovering small-disjunct rules in data mining

Abstract

This paper addresses the well-known classification task of data mining, where the goal is to discover rules predicting the class of examples (records of a given data set). In the context of data mining, small disjuncts are rules covering a small number of examples. Hence, these rules are usually error-prone, which contributes to a decrease in predictive accuracy. At first glance, this is not a serious problem, since the impact on predictive accuracy should be small. However, although each small disjunct covers few examples, the set of all small disjuncts can cover a large number of examples. This paper presents evidence that this is the case in several data sets. This paper also addresses the problem of small disjuncts by using a hybrid decision-tree/genetic algorithm approach. In essence, examples belonging to large disjuncts are classified by rules produced by a decision-tree algorithm (C4.5), while examples belonging to small disjuncts are classified by a genetic algorithm specifically designed for discovering small-disjunct rules. We present results comparing the predictive accuracy of this hybrid system with the prediction accuracy of three versions of C4.5 alone in eight public domain data sets. Overall, the results show that our hybrid system achieves better predictive accuracy than all three versions of C4.5 alone.

A hybrid decision tree/genetic algorithm method for data mining

“…This is a hybrid method that combines decision trees and genetic algorithms [5], [6], [3], [4]. The basic idea is to use a decision-tree algorithm to classify examples belonging to large disjuncts and use a genetic algorithm to discover rules classifying examples belonging to small disjuncts.…”

Evolutionary Induction of Descriptive Rules in a Market Problem

“…• Carvalho and Freitas present in [8,9] a hybrid decision tree/genetic algorithm approach for a predictive rule induction process, which follows the following model: examples belonging to large disjuncts are classified by rules produced by a decision-tree algorithm while examples belonging to small disjuncts are classified by rules produced by a GA specifically designed for this task. In the GA each individual represents a small-disjunct rule identified by a decision tree leaf node.…”

“…The GA uses a new operator specially designed to improve the comprehensibility of the rules. The authors propose in [10] some modifications to the original GA. The most significant change is the use of a sequential niching method [7] to foster population diversity and avoid the GA convergence to a single rule.…”

“…In this GA, developed by Kwedlo and Kretowski [38], a single chromosome encodes a set of rules in a variable length genome. The consequent is not coded, it remains fixed for each iteration of the GA (as in [8,9,10,22]). As well as the standard genetic operators it includes four specific for rule induction: one operator which modifies a chromosome in order to include an additional positive example, another which modifies the chromosome in order to eliminate a negative example, a rule elimination operator and a rule inclusion operator.…”

Genetic Algorithm to Solve the Problem of Small Disjunct In the Decision Tree Based Intrusion Detection System

“…The large disjunct are the rule sets which covers a large number of training instances and small disjuncts are the rules sets which covers small number of the training instances. The rule induction methods ignores small disjunct and discovers only large disjunct therefore the classification accuracy will significantly degraded in many circumstances [8,9,10].…”