Unpack Local Model Interpretation for GBDT

Abstract

A gradient boosting decision tree (GBDT), which aggregates a collection of single weak learners (i.e. decision trees), is widely used for data mining tasks. Because GBDT inherits the good performance from its ensemble essence, much attention has been drawn to the optimization of this model. With its popularization, an increasing need for model interpretation arises. Besides the commonly used feature importance as a global interpretation, feature contribution is a local measure that reveals the relationship between a specific instance and the related output. This work focuses on the local interpretation and proposes an unified computation mechanism to get the instance-level feature contributions for GBDT in any version. Practicality of this mechanism is validated by the listed experiments as well as applications in real industry scenarios.

Improved Random Forest Algorithm Based on Decision Paths for Fault Diagnosis of Chemical Process with Incomplete Data

“…The proportion of positive samples in all training samples contained in this node is denoted as r t k (y), which can also be considered as the probability that the training sample contained in node k belongs to the predicted sample category y. The difference in proportion of positive samples in child node and its corresponding parent node can be viewed as the node importance of the child node [40][41][42]. The larger the difference, the higher the purity of the sample split to the child node compared to that of the parent node, thus the higher the importance of the child node for the classification problem.…”

Adapted Tree Boosting for Transfer Learning

“…GBDT is a common choice in machine learning tasks. Besides the high performance and efficiency, GBDT and its variants also provides the model interpretability [47] and the easiness of parameter tuning. The most direct transfer is first train a model on the source dataset.…”

Interpretable MTL from Heterogeneous Domains using Boosted Tree

“…The interpretability of boosted tree model in both global and local level has been shown in [3]. In our work, since the whole model of each task consists of the common part and the specific part, so we collect them all to get the whole importance of each feature.…”

“…In our work, since the whole model of each task consists of the common part and the specific part, so we collect them all to get the whole importance of each feature. For each instance, the contribution of each feature to the final prediction can be calculated with the method in [3]. An example of the top 20 important feature in task2 of Scene1 is shown in figure 2.…”

“…(2) The construction of the trees in the common model may be unbeneficial or even harmful for some task after some rounds, which means that a mechanism is needed to find the proper round so that a task can quit from the common model training process if necessary. (3) The training of the second stage should take the information of the first stage into consideration, so that the obtained model can be more effective, instead of simply combining two boosted tree models when predicting for each task. To handle these problems, a regularization strategy is proposed for the construction of each tree to alleviate the domination problem, and an early stopping strategy is designed so that a task can quit the common process if further training will not improve its performance.…”