A Survey on the Robustness of Feature Importance and Counterfactual Explanations

Mishra, Sunita; Dutta, Sanghamitra; Long, Jing; Magazzeni, Daniele

doi:10.48550/arxiv.2111.00358

Cited by 8 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [22], the focus is on analytical trade-offs between validity and cost. We also refer to [23] for a survey on the robustness of both feature-based attributions and counterfactuals.…”

Section: Methodsmentioning

confidence: 99%

Robust Counterfactual Explanations for Tree-Based Ensembles

Dutta¹,

Long²,

Mishra³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Counterfactual explanations inform ways to achieve a desired outcome from a machine learning model. However, such explanations are not robust to certain real-world changes in the underlying model (e.g., retraining the model, changing hyperparameters, etc.), questioning their reliability in several applications, e.g., credit lending. In this work, we propose a novel strategy -that we call RobX -to generate robust counterfactuals for tree-based ensembles, e.g., XGBoost. Tree-based ensembles pose additional challenges in robust counterfactual generation, e.g., they have a non-smooth and non-differentiable objective function, and they can change a lot in the parameter space under retraining on very similar data. We first introduce a novel metric -that we call Counterfactual Stability -that attempts to quantify how robust a counterfactual is going to be to model changes under retraining, and comes with desirable theoretical properties. Our proposed strategy RobX works with any counterfactual generation method (base method) and searches for robust counterfactuals by iteratively refining the counterfactual generated by the base method using our metric Counterfactual Stability. We compare the performance of RobX with popular counterfactual generation methods (for tree-based ensembles) across benchmark datasets. The results demonstrate that our strategy generates counterfactuals that are significantly more robust (nearly 100% validity after actual model changes) and also realistic (in terms of local outlier factor) over existing state-of-the-art methods.How do we generate counterfactuals for tree-based ensembles that are not only close but also robust to changes in the model?

show abstract

“…In [22], the focus is on analytical trade-offs between validity and cost. We also refer to [23] for a survey on the robustness of both feature-based attributions and counterfactuals.…”

Section: Methodsmentioning

confidence: 99%

Robust Counterfactual Explanations for Tree-Based Ensembles

Dutta¹,

Long²,

Mishra³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Various local explanation methods however have been criticized for not being robust (Artelt et al, 2021;Hancox-Li, 2020;Mishra et al, 2021) or that they might fail to explain the global behavior of complex models (Slack et al, 2021).…”

Section: Counterfactual Explanationsmentioning

confidence: 99%

"How to make them stay?" -- Diverse Counterfactual Explanations of Employee Attrition

Artelt¹,

Gregoriades²

2023

Preprint

View full text Add to dashboard Cite

Employee attrition is an important and complex problem that can directly affect an organisation's competitiveness and performance. Explaining the reasons why employees leave an organisation is a key human resource management challenge due to the high costs and time required to attract and keep talented employees. Businesses therefore aim to increase employee retention rates to minimise their costs and maximise their performance. Machine learning (ML) has been applied in various aspects of human resource management including attrition prediction to provide businesses with insights on proactive measures on how to prevent talented employees from quitting. Among these ML methods, the best performance has been reported by ensemble or deep neural networks, which by nature constitute black box techniques and thus cannot be easily interpreted. To enable the understanding of these models' reasoning several explainability frameworks have been proposed to either explain individual cases using local interpretation approaches or provide global explanations describing the overall logic of the predictive model. Counterfactual explanation methods have attracted considerable attention in recent years since they can be used to explain and recommend actions to be performed to obtain the desired outcome. However current counterfactual explanations methods focus on optimising the changes to be made on individual cases to achieve the desired outcome. In the attrition problem it is important to be able to foresee what would be the effect of an organisation's action to a group of employees where the goal is to prevent them from leaving the company. Therefore, in this paper we propose the use of counterfactual explanations focusing on multiple attrition cases from historical data, to identify the optimum interventions that an organisation needs to make to its practices/policies to prevent or minimise attrition probability for these cases. The proposed technique is applied on an employee attrition dataset, used to train binary classifiers. Counterfactual explanations are generated based on multiple attrition cases, thus, providing recommendations to the human resource department on how to prevent attrition .

show abstract

“…Evaluate consistency among explanations provided by multiple methods at global/local stage is a straightforward and inexpensive approach to get insights into model stability and robustness, but results must be handled cautiously. Empirical and theoretical analysis demonstrated that the majority of popular feature importance and counterfactual explanation methods are non-robust (Mishra et al 2021). In particular, most works focused on XAI methods that are specific to DNN models.…”

Section: Stability and Robustnessmentioning

confidence: 99%

Explainable AI for clinical and remote health applications: a survey on tabular and time series data

Martino

Delmastro

2022

Artif Intell Rev

View full text Add to dashboard Cite

Nowadays Artificial Intelligence (AI) has become a fundamental component of healthcare applications, both clinical and remote, but the best performing AI systems are often too complex to be self-explaining. Explainable AI (XAI) techniques are defined to unveil the reasoning behind the system’s predictions and decisions, and they become even more critical when dealing with sensitive and personal health data. It is worth noting that XAI has not gathered the same attention across different research areas and data types, especially in healthcare. In particular, many clinical and remote health applications are based on tabular and time series data, respectively, and XAI is not commonly analysed on these data types, while computer vision and Natural Language Processing (NLP) are the reference applications. To provide an overview of XAI methods that are most suitable for tabular and time series data in the healthcare domain, this paper provides a review of the literature in the last 5 years, illustrating the type of generated explanations and the efforts provided to evaluate their relevance and quality. Specifically, we identify clinical validation, consistency assessment, objective and standardised quality evaluation, and human-centered quality assessment as key features to ensure effective explanations for the end users. Finally, we highlight the main research challenges in the field as well as the limitations of existing XAI methods.

show abstract

A Survey on the Robustness of Feature Importance and Counterfactual Explanations

Cited by 8 publications

References 23 publications

Robust Counterfactual Explanations for Tree-Based Ensembles

Robust Counterfactual Explanations for Tree-Based Ensembles

"How to make them stay?" -- Diverse Counterfactual Explanations of Employee Attrition

Explainable AI for clinical and remote health applications: a survey on tabular and time series data

Contact Info

Product

Resources

About