On the Trustworthiness of Tree Ensemble Explainability Methods

Abstract

The recent increase in the deployment of machine learning models in critical domains such as healthcare, criminal justice, and finance has highlighted the need for trustworthy methods that can explain these models to stakeholders. Feature importance methods (e.g. gain and SHAP) are among the most popular explainability methods used to address this need. For any explainability technique to be trustworthy and meaningful, it has to provide an explanation that is accurate and stable. Although the stability of local feature importance methods (explaining individual predictions) has been studied before, there is yet a knowledge gap about the stability of global features importance methods (explanations for the whole model). Additionally, there is no study that evaluates and compares the accuracy of global feature importance methods with respect to feature ordering. In this paper, we evaluate the accuracy and stability of global feature importance methods through comprehensive experiments done on simulations as well as four real-world datasets. We focus on tree-based ensemble methods as they are used widely in industry and measure the accuracy and stability of explanations under two scenarios: 1. when inputs are perturbed 2. when models are perturbed. Our findings provide a comparison of these methods under a variety of settings and shed light on the limitations of global feature importance methods by indicating their lack of accuracy with and without noisy inputs, as well as their lack of stability with respect to: 1.

Evaluating Explanation Correctness in Legal Decision Making

“…There are also minor fluctuations in SHAP and LIME, although they are visually smoother. Similar to findings from existing work [10], the faithfulness of these XAI methods are inconsistent on decision tree models. The inter-rater agreements for the survey questions are reported in Table 4.…”

“…a decrease in accuracy or change in polarity) [6] [9]. There are many variations -for example, it is also possible to evaluate faithfulness with synthetic data and known feature importance [10].…”

Implementing local-explainability in Gradient Boosting Trees: Feature Contribution

“…Because of the importance of explicability in machine learning models and the success of GBDT as a predictive model, SHAP explicability have been used many applications and studies. However the SHAP explicability over the GBDT seem to have lack of accuracy and stability in their explicability values [29,30,31].…”