Semi-supervised self-training for decision tree classifiers

Abstract

We consider semi-supervised learning, learning task from both labeled and unlabeled instances and in particular, self-training with decision tree learners as base learners. We show that standard decision tree learning as the base learner cannot be effective in a self-training algorithm to semi-supervised learning. The main reason is that the basic decision tree learner does not produce reliable probability estimation to its predictions. Therefore, it cannot be a proper selection criterion in self-training. We consider the effect of several modifications to the basic decision tree learner that produce better probability estimation than using the distributions at the leaves of the tree. We show that these modifications do not produce better performance when used on the labeled data only, but they do benefit more from the unlabeled data in self-training. The modifications that we consider are Naive Bayes Tree, a combination of No-pruning and Laplace correction, grafting, and using a distance-based measure. We then extend this improvement to algorithms for ensembles of decision trees and we show that the ensemble learner gives an extra improvement over the adapted decision tree learners.

Deep Neural Network Self-training Based on Unsupervised Learning and Dropout

“…Tanha et al [2] proposed a decision tree based self-training approach. This approach uses a label proportion of training labeled examples included in corresponding leaf node as confidence measure of classified example.…”

“…For example, a selftraining decision tree [2] has used class proportion of leaf as confidence measure. A self-training support vector machine (SVM) [3] have used distance between examples and the classification boundary.…”

Production planning of Pakistan Tobacco Company (PTC) using quantitative and multiple‐criteria decision analysis—A case in‐point

“…Another advantage of the decision tree is that it is easy to understand and manipulate by the manufacturing engineer (Perez, Datta-Gupta, & Misra, 2005) including C-4.5 (Quinlan, 1993), QUEST (Agbon, Aldana, & Araque, 2003), ID-3 (Papagelis & Kalles, 2000), and GA-Tree (Quinlan, 1987). Strategies for similar capabilities decision (Loh & Shih, 1997), which manage the overfitting issue, have been proposed by Quinlan (1986), and the matter of the capacity to be made greater or lesser has been discussed by Tanha, Someren, and Afsarmanesh (2015). This helps in choosing a decision from multiple choices with the same capability on the basis of the probability of occurrence (Chandra & Varghese, 2009).…”

Self-Trained LMT for Semisupervised Learning

“…Tanha et al [10] suggested that using decision tree classifiers as base classifiers along with self-training algorithm is not quite effective as semisupervised learning is concerned mainly due to low performance when decision tree classifiers compute probability estimations for their predictions. However, decision trees are not demanding in training time and produce easily comprehensive models.…”

“…A series of modifications have been proposed so as to refrain from using the simplistic proportion distribution at the leaves of a pruned decision tree [11]. Laplacian correction and grafted decision trees are some of them [10]. Torgo [12] also made a thorough study of tree-based regression models and focused on generation of tree models and on pruning by tree selection.…”