Building Decision Trees for the Multi-class Imbalance Problem

Abstract

Abstract. Learning in imbalanced datasets is a pervasive problem prevalent in a wide variety of real-world applications. In imbalanced datasets, the class of interest is generally a small fraction of the total instances, but misclassification of such instances is often expensive. While there is a significant body of research on the class imbalance problem for binary class datasets, multi-class datasets have received considerably less attention. This is partially due to the fact that the multi-class imbalance problem is often much harder than its related binary class problem, as the relative frequency and cost of each of the classes can vary widely from dataset to dataset. In this paper we study the multi-class imbalance problem as it relates to decision trees (specifically C4.4 and HDDT), and develop a new multi-class splitting criterion. From our experiments we show that multi-class Hellinger distance decision trees, when combined with decomposition techniques, outperform C4.4.

Learning Imbalanced Multi-class Data with Optimal Dichotomy Weights

“…In each dichotomy of ECOC, HDDT [8] is used for imbalanced binary-class subtask, which is a decision tree method with a novel splitting criteria for imbalanced binary-class data. The method is extended to a multi-class method MC-HDDT [16]. Since their experiments showed ECOC + HDDT performs better than MC-HDDT, we do not compare with MC-HDDT in addition.…”

“…There are a few studies [12], [16], [19], [21], [26], [27], [29] on multi-class imbalance. Most of them were published very recently [12], [16], [19], [29].…”

“…Most of them were published very recently [12], [16], [19], [29]. [26] uses genetic algorithm to determine the optimal cost vector and exploits a multiclass cost-sensitive learning method.…”

“…[26] uses genetic algorithm to determine the optimal cost vector and exploits a multiclass cost-sensitive learning method. MC-HDDT [16] is a decision tree method using a multi-class splitting criterion in favor of the minority classes. DyS [19] is a neural network dynamically sampling data to update the weights during the learning procedure.…”

“…There are also higher degree of concept complexity, more small disjuncts, and even higher imbalance rate (the size of the largest class divided by the size of the smallest class). There are a few work studied multi-class imbalance [12], [16], [19], [21], [26], [27], [29], and they will be briefly reviewed in Section II.…”