Geometric Heuristics for Transfer Learning in Decision Trees

Abstract

Motivated by a network fault detection problem, we study how recall can be boosted in a decision tree classifier, without sacrificing too much precision. This problem is relevant and novel in the context of transfer learning (TL), in which few target domain training samples are available. We define a geometric optimization problem for boosting the recall of a decision tree classifier, and show it is NP-hard. To solve it efficiently, we propose several near-linear time heuristics, and experimentally validate these heuristics in the context of TL. Our evaluation includes 7 public datasets, as well as 6 network fault datasets, and we compare our heuristics with several existing TL algorithms, as well as exact mixed integer linear programming (MILP) solutions to our optimization problem. We find that our heuristics boost recall in a manner similar to optimal MILP solutions, yet require several orders of magnitude less compute time. In many cases the 𝐹 1 score of our approach is competitive, and often better, than other TL algorithms. Moreover, our approach can be used as a building block to apply transfer learning to more powerful ensemble methods, such as random forests. CCS CONCEPTS• Computing methodologies → Classification and regression trees; Transfer learning.

Teaching reform and innovation of vocational development and employment guidance courses in colleges and universities based on random forest model

“…If the attribute value is continuous, it must first be discrete by processing it to make it discrete. The second stage of decision tree optimization is to cut out the isolated points and noise generated in the training data so that the constructed decision tree can be compared with the values of each attribute of the sample data to achieve the purpose of classifying and mining the unknown sample data [32][33]. The generation process of the decision tree algorithm is shown in Figure 2.…”