Bloomy Decision Tree for Multi-objective Classification

Abstract

This paper presents a novel decision-tree induction for a multi-objective data set, i.e. a data set with a multi-dimensional class. Inductive decision-tree learning is one of the frequently-used methods for a single-objective data set, i.e. a data set with a single-dimensional class. However, in a real data analysis, we usually have multiple objectives, and a classifier which explains them simultaneously would be useful and would exhibit higher readability. A conventional decision-tree inducer requires transformation of a multi-dimensional class into a singledimensional class, but such a transformation can considerably worsen both accuracy and readability. In order to circumvent this problem we propose a bloomy decision tree which deals with a multi-dimensional class without such transformations. A bloomy decision tree has a set of split nodes each of which splits examples according to their attribute values, and a set of flower nodes each of which predicts a class dimension of examples. A flower node appears not only at the fringe of a tree but also inside a tree. Our pruning is executed during tree construction, and evaluates each class dimension based on Cramér's V. The proposed method has been implemented as D3-B (Decision tree in Bloom), and tested with eleven data sets. The experiments showed that D3-B has higher accuracies in nine data sets than C4.5 and tied with it in the other two data sets. In terms of readability, D3-B has a smaller number of split nodes in all data sets, and thus outperforms C4.5.

Learning Predictive Clustering Rules

“…However, it extends rule induction to the more general task of multi-objective prediction. While some work exists on multi-objective classification with decision trees (e.g., [16]), the authors are not aware of any work on rule-induction for multi-objective classification. Also, little work exists on rule-based regression (some recent examples come from the are of ILP, e.g., FORS [8]), let alone rule-based multi-objective regression (or multi-objective prediction in general, with mixed continuous and discrete targets).…”

Iterative classification for multiple target attributes

“…In addition, the set of feature variables connects with the set of class variables through a bi-partite directed graph. Following the same line of research, Suzuki et al (2001) extended a decision tree to simultaneously explain multiple labels; Last (2004) presented a Multi-objective Info-Fuzzy Network.…”

Ensembles of Multi-Objective Decision Trees

“…This task is known under the name of multi-objective prediction. Existing learning techniques have been extended to address this task by learning to predict all target attributes at once [1,2,3,4]. This has two main advantages over building a separate model for each target: first, a multi-objective model is usually much smaller than the total size of the individual models for all target attributes, and second, such a multi-objective model explicates dependencies between the different target attributes.…”