HHCART: An oblique decision tree

Abstract

Decision trees are a popular technique in statistical data classification. They recursively partition the feature space into disjoint sub-regions until each sub-region becomes homogeneous with respect to a particular class. The basic Classification and Regression Tree (CART) algorithm partitions the feature space using axis parallel splits. When the true decision boundaries are not aligned with the feature axes, this approach can produce a complicated boundary structure. Oblique decision trees use oblique decision boundaries to potentially simplify the boundary structure. The major limitation of this approach is that the tree induction algorithm is computationally expensive. In this article we present a new decision tree algorithm, called HHCART. The method utilizes a series of Householder matrices to reflect the training data at each node during the tree construction. Each reflection is based on the directions of the eigenvectors from each classes' covariance matrix. Considering axis parallel splits in the reflected training data provides an efficient way of finding oblique splits in the unreflected training data. Experimental results show that the accuracy and size of the HHCART trees are comparable with some benchmark methods in the literature. The appealing feature of HHCART is that it can handle both qualitative and quantitative features in the same oblique split.

A reflected feature space for CART

“…Reflecting the feature vectors in this way makes d ik parallel to e and provides a simple and effective way to find oblique splits (Robertson, Price & Reale ; Wickramarachchi et al . ). The authors propose two HHCART methods.…”

“…HHCART (Wickramarachchi et al . ) is another oblique decision tree. Rather than searching for oblique splits directly, HHCART finds the best axis parallel split in a set of reflected feature spaces.…”

“…However, oblique trees are less interpretable and finding an optimal split can be computationally demanding (Wickramarachchi et al . ; Hastie, Tibshirani & Friedman , Section 9.2).…”

“…The following section describes two decision trees, Geometric Decision Tree (GDT; Manwani & Sastry ) and HHCART (Wickramarachchi et al . ), that are pertinent to this paper. Section 3 introduces a modified GDT angle bisector that is used to extend HHCART to produce a new classification tree, called HHCART(G).…”

On Multi-Scale Entropy Analysis of Order-Tracking Measurement for Bearing Fault Diagnosis under Variable Speed

“…The merits of the decision tree classifier includes that the classification rule is simple and less computation effort is required. The algorithms of the decision trees that have been utilized broadly include ID3 [21], C4.5 [22], CHAID [23], CART [24], and QUEST [25]. Due to its flexible capability for continuous and discrete data processing, the C4.5 algorithm [22] was employed in this research to construct the decision tree model for the classification of different bearing defects.…”