Lexicographic multi-objective evolutionary induction of decision trees

Abstract

Among the several tasks that evolutionary algorithms have successfully employed, the induction of classification rules and decision trees has been shown to be a relevant approach for several application domains. Decision tree induction algorithms represent one of the most popular techniques for dealing with classification problems. However, conventionally used decision trees induction algorithms present limitations due to the strategy they usually implement: recursive top-down data partitioning through a greedy split evaluation. The main problem with this strategy is quality loss during the partitioning process, which can lead to statistically insignificant rules. In this paper, we propose a new GA-based algorithm for decision tree induction. The proposed algorithm aims to prevent the greedy strategy and to avoid converging to local optima. For such, it is based on a lexicographic multi-objective approach. In order to evaluate the proposed algorithm, it is compared with a well-known and frequently used decision tree induction algorithm using different public datasets. According to the experimental results, the proposed algorithm is able to avoid the previously described problems, reporting accuracy gains. Even more important, the proposed algorithm induced models with a significantly reduction in the complexity considering tree sizes.

A Survey of Evolutionary Algorithms for Decision-Tree Induction

“…Although solution encoding can differentiate between GAs and GP, the main question is not what the representation is (e.g. a linear string or a tree) but rather how the representation is interpreted [28].…”

“…Furthermore, they affirm that it is natural to use a tree structure to represent decision trees and that the mutation-crossover operators can be efficiently altered to match this structure. For other tree-encoding scheme examples, see [22], [28], [40]- [47].…”

Evolutionary model tree induction

“…It is closely related to LEGAL-Tree [2], [3], which induces decision trees through the evolutionary paradigm. We have extended such approach for dealing with regression problems, in order to generate efficient and comprehensible regression models for a variety of data mining applications.…”

“…Evolutionary induction of decision trees is a well-addressed issue, as presented in works such as [2], [3], [14], [26]. However, very few studies have discussed the evolutionary induction of regression/model trees, and we briefly review them below.…”

“…Evolutionary induction of decision trees for classification tasks is well-explored in the research community. For instance, Basgalupp et al [3] proposed an evolutionary algorithm for the induction of decision trees, named LEGAL-Tree, which looks for a good trade-off between accuracy and model comprehensibility. Very few works however propose evolving regression/model trees in order to avoid the problems previously mentioned.…”