Evolutionary Design of Decision-Tree Algorithms Tailored to Microarray Gene Expression Data Sets

Abstract

Decision-tree induction algorithms are widely used in machine learning applications in which the goal is to extract knowledge from data and present it in a graphically intuitive way. The most successful strategy for inducing decision trees is the greedy top-down recursive approach, which has been continuously improved by researchers over the past 40 years. In this paper, we propose a paradigm shift in the research of decision trees: instead of proposing a new manually designed method for inducing decision trees, we propose automatically designing decision-tree induction algorithms tailored to a specific type of classification data set (or application domain). Following recent breakthroughs in the automatic design of machine learning algorithms, we propose a hyper-heuristic evolutionary algorithm called hyper-heuristic evolutionary algorithm for designing decision-tree algorithms (HEAD-DT) that evolves design components of top-down decision-tree induction algorithms. By the end of the evolution, we expect HEAD-DT to generate a new and possibly better decision-tree algorithm for a given application domain. We perform extensive experiments in 35 real-world microarray gene expression data sets to assess the performance of HEAD-DT, and compare it with very well known decisiontree algorithms such as C4.5, CART, and REPTree. Results show that HEAD-DT is capable of generating algorithms that significantly outperform the baseline manually designed decisiontree algorithms regarding predictive accuracy and F-measure.

Evolving decision-tree induction algorithms with a multi-objective hyper-heuristic

“…The work of Basgalupp et al [8] proposes a hyper-heuristic based on grammatical evolution to automatically design split criteria in decision-tree induction, generating novel criteria instead of selecting them as in [22]. Finally, the work of Barros et al [4,5,6] proposes the evolution of complete decision-tree induction algorithms through a single-objective EA called HEAD-DT. In this paper, we extend HEAD-DT so it can deal with multiple conflicting objectives, generating algorithms capable of providing both accurate and compact decision trees.…”

“…The proposed system is called MOHEAD-DT (Multi-Objective Hyper-Heuristic Evolutionary Algorithm for Automatically Designing Decision-Tree Algorithms), and it is a thorough extension of HEAD-DT [5,6], which is presented in greater detail in the next subsection. MOHEAD-DT allows for a trade-off between predictive performance and model comprehensibility, the latter being of great importance in several application domains [15].…”

“…MOHEAD-DT allows for a trade-off between predictive performance and model comprehensibility, the latter being of great importance in several application domains [15]. Barros et al [4,5,6] have proposed HEAD-DT (Hyper-Heuristic Evolutionary Algorithm for Automatically Designing Decision-Tree Algorithms), which is a hyper-heuristic single-objective EA capable of automatically designing complete top-down decision-tree induction algorithms. It can be regarded as an alternative to the manual design of such algorithms, in a process called bias-fitting algorithmic generation.…”

“…The latter was applied with the goal of evolving a decision-tree algorithm tailored to a particular domain, e.g., flexible-receptor docking data [7], software effort prediction [9], and gene expression data classification [6]. In the specific framework, the part of the dataset that belongs to the meta-training set is further divided into training and validation, with typical values being 70% for training and 30% for validation.…”

“…Recently, Barros et al [2,4,5] have proposed a paradigm shift, and instead of evolving decision trees for individual datasets, they proposed the automatic design of decision-tree induction algorithms tailored to specific application domains such as software effort estimation [9], classification of molecular docking data [7], and cancer classification based on the levels of gene expression [6]. For such, the authors developed a single-objective hyper-heuristic EA that evolved a set of linear-genome-based individuals throughout a fixed number of generations, namely HEAD-DT.…”