Interpreting Black Box Models via Hypothesis Testing

Burns, Collin; Thomason, Jesse; Tansey, Wesley

doi:10.48550/arxiv.1904.00045

Cited by 7 publications

(7 citation statements)

References 16 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4) Randomization and Feature Testing: The Interpretability Randomization Test (IRT) and the One-Shot Feature Test (OSFT) introduced by Burns et al [55] focuses on discovering important features by replacing the features with uninformative counterfactuals. Modeling the feature replacement with a hypothesis testing framework, the authors illustrate an interesting way to examine contextual importance.…”

Section: A Perturbation-basedmentioning

confidence: 99%

Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey

Das,

Rad

2020

Preprint

176

187

View full text Add to dashboard Cite

Nowadays, deep neural networks are widely used in mission critical systems such as healthcare, self-driving vehicles, and military which have direct impact on human lives. However, the black-box nature of deep neural networks challenges its use in mission critical applications, raising ethical and judicial concerns inducing lack of trust. Explainable Artificial Intelligence (XAI) is a field of Artificial Intelligence (AI) that promotes a set of tools, techniques, and algorithms that can generate high-quality interpretable, intuitive, human-understandable explanations of AI decisions. In addition to providing a holistic view of the current XAI landscape in deep learning, this paper provides mathematical summaries of seminal work. We start by proposing a taxonomy and categorizing the XAI techniques based on their scope of explanations, methodology behind the algorithms, and explanation level or usage which helps build trustworthy, interpretable, and self-explanatory deep learning models. We then describe the main principles used in XAI research and present the historical timeline for landmark studies in XAI from 2007 to 2020. After explaining each category of algorithms and approaches in detail, we then evaluate the explanation maps generated by eight XAI algorithms on image data, discuss the limitations of this approach, and provide potential future directions to improve XAI evaluation.

show abstract

Section: A Perturbation-basedmentioning

confidence: 99%

Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey

Das,

Rad

2020

Preprint

176

187

View full text Add to dashboard Cite

show abstract

“…In contrast to LIME which only locally explains, the SHAP method can be used for global explanation. There are also alternative methods which include influence functions [32], a multiple hypothesis testing framework [33], and many other methods.…”

Section: Related Workmentioning

confidence: 99%

SurvNAM: The machine learning survival model explanation

Utkin¹,

Satyukov²,

Konstantinov³

2021

Preprint

View full text Add to dashboard Cite

A new modification of the Neural Additive Model (NAM) called SurvNAM and its modifications are proposed to explain predictions of the black-box machine learning survival model. The method is based on applying the original NAM to solving the explanation problem in the framework of survival analysis. The basic idea behind SurvNAM is to train the network by means of a specific expected loss function which takes into account peculiarities of the survival model predictions and is based on approximating the black-box model by the extension of the Cox proportional hazards model which uses the well-known Generalized Additive Model (GAM) in place of the simple linear relationship of covariates. The proposed method SurvNAM allows performing the local and global explanation. A set of examples around the explained example is randomly generated for the local explanation. The global explanation uses the whole training dataset. The proposed modifications of SurvNAM are based on using the Lasso-based regularization for functions from GAM and for a special representation of the GAM functions using their weighted linear and non-linear parts, which is implemented as a shortcut connection. A lot of numerical experiments illustrate the SurvNAM efficiency.

show abstract

“…Another very popular method is the SHAP [49] which takes a game-theoretic approach for optimizing a regression loss function based on Shapley values [33]. Alternative methods are influence functions [31], a multiple hypothesis testing framework [9], and many other methods.…”

Section: Related Workmentioning

confidence: 99%

SurvLIME: A method for explaining machine learning survival models

Kovalev¹,

Utkin²,

Kasimov³

2020

Preprint

View full text Add to dashboard Cite

A new method called SurvLIME for explaining machine learning survival models is proposed. It can be viewed as an extension or modification of the well-known method LIME. The main idea behind the proposed method is to apply the Cox proportional hazards model to approximate the survival model at the local area around a test example. The Cox model is used because it considers a linear combination of the example covariates such that coefficients of the covariates can be regarded as quantitative impacts on the prediction. Another idea is to approximate cumulative hazard functions of the explained model and the Cox model by using a set of perturbed points in a local area around the point of interest. The method is reduced to solving an unconstrained convex optimization problem. A lot of numerical experiments demonstrate the SurvLIME efficiency.

show abstract

Interpreting Black Box Models via Hypothesis Testing

Cited by 7 publications

References 16 publications

Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey

Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey

SurvNAM: The machine learning survival model explanation

SurvLIME: A method for explaining machine learning survival models

Contact Info

Product

Resources

About