A Classifier Based on a Decision Tree with Verifying Cuts

Abstract

This article introduces a new method of a decision tree construction. Such construction is performed using additional cuts applied for a verification of the cuts' quality in tree nodes during the classification of objects. The presented approach allows us to exploit the additional knowledge represented in the attributes which could be eliminated using greedy methods. The paper includes the results of experiments performed on data sets from a biomedical database and machine learning repositories. In order to evaluate the presented method, we compared its performance with the classification results of a local discretization decision tree, well known from literature. Our new method outperforms the existing method, which is also confirmed by statistical tests.

Verifying cuts as a tool for improving a classifier based on a decision tree

“…As in the previous article [1], the motivation for our work concerns the validity of classical approach used to data sets with large number of attributes. We recall that the method chooses only one split (for a single attribute) with the best quality based on the selected measure, at the given step of searching for optimal binary partitions.…”

“…In such case, the method would greedily eliminate the information contained in attributes, which are similar in terms of quality of potential cuts, but are different with respect to domain knowledge, which they represent. The main idea presented in [1] is based on the fact that at a given stage of searching for partitions of a set of attributes, the family of k-binary verifying partitions is determined after construction of the optimal binary partition of a set of objects. Obviously, it refers to family of partitions which are similar to the optimal partition and concerns other attributes than the attributes used in the optimal partition.…”

“…They are additional cuts, which enable to evaluate the quality of cuts in tree nodes during classification of objects. Our experiments conducted on real data and described in [1] have shown that data with numerous set of attributes constitutes a class of data where v-tree outperforms the conventional approach. However, only one technique of choosing verifying cuts was tested, namely the one based on the maximization of the number of discerned object pairs with different decision class membership.…”

“…In this paper two another techniques of verifying cuts construction based on the Entropy and the Gini's Index are tested. Furthermore, we conducted comparative experiments using these two approaches and the one described in [1].…”

“…ECISION tree with verifying cuts [1] (denoted by vtree) is a method of decision tree construction formed in response to the problem of classification data with a large number of attributes. Such data can contain a lot of attributes that bear similarity with respect to the quality of potential cuts but significantly different with respect to domain knowledge represented.…”