Confidence Decision Trees via Online and Active Learning for Streaming Data

Abstract

Decision tree classifiers are a widely used tool in data stream mining. The use of confidence intervals to estimate the gain associated with each split leads to very effective methods, like the popular Hoeffding tree algorithm. From a statistical viewpoint, the analysis of decision tree classifiers in a streaming setting requires knowing when enough new information has been collected to justify splitting a leaf. Although some of the issues in the statistical analysis of Hoeffding trees have been already clarified, a general and rigorous study of confidence intervals for splitting criteria is missing. We fill this gap by deriving accurate confidence intervals to estimate the splitting gain in decision tree learning with respect to three criteria: entropy, Gini index, and a third index proposed by Kearns and Mansour. Our confidence intervals depend in a more detailed way on the tree parameters. We also extend our confidence analysis to a selective sampling setting, in which the decision tree learner adaptively decides which labels to query in the stream. We furnish theoretical guarantee bounding the probability that the classification is nonoptimal learning the decision tree via our selective sampling strategy. Experiments on real and synthetic data in a streaming setting show that our trees are indeed more accurate than trees with the same number of leaves generated by other techniques and our active learning module permits to save labeling cost. In addition, comparing our labeling strategy with recent methods, we show that our approach is more robust and consistent respect all the other techniques applied to incremental decision trees. 1

Combining Federated and Active Learning for Communication-efficient Distributed Failure Prediction in Aeronautics

“…This limit case is more frequent for small numbers of clients. Beside this, we can observe that the accuracy is stable at about 61% which is an excellent level of performance comparable to state-of-the-art results in the non-distributed case with [16] reporting about 62% of accuracy with a similar request budget. This validates the potential of the approach we propose for communication-efficient distributed learning.…”

“…Examples of this approach can be found in [14] for an application to document classification and in [15] for an application to sentiment analysis. [16] is an article of particular interest to this article as it proposes a new way to train decision trees using an active learning approach that focuses on minimizing the risk of selecting a sub-optimal split when a new leaf is added to the tree by computing confidence intervals. To put it differently, when a new leaf is added to the tree, the comparison is made between the ideal decision tree that could have been made if the true distribution of the samples were known and the expected performance of the current tree.…”

“…The approach we propose to combine Active Learning and Federated Learning is to use the confidence based active online Decision Tree (DT) from [16] as a base model that is learned on the client side and sent to the server. The server aggregates the client DTs in an ensemble model similar to a RF to learn an accurate model without accessing the raw data.…”

“…From the clients perspective, they have at their disposal a stream of observations to train their local DT. As described in [16], whenever the uncertainty surrounding a new split in the DT exceeds a certain confidence threshold, a request is made to reveal the true label of the samples needed to decide on the right split and a part of the request budget is spent. Here the requests are sent to the server and the labels are decided using the ensemble model.…”

“…In order to enable a meaningful comparison with other works, we have selected for our experiments the MOA airlines dataset 1 , one of the public benchmarks that was used in [16]. Though it is related to the air transport industry, it is not strictly speaking a dataset from an aeronautical system but it presents the advantages of being public and enabling comparisons with other solutions.…”