Understanding surrogate explanations: the interplay between complexity, fidelity and coverage

Poyiadzi, Rafael; Renard, Xavier; Laugel, Thibault; Santos-Rodríguez, Raúl; Detyniecki, Marcin

doi:10.48550/arxiv.2107.04309

Cited by 3 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The process of deriving surrogate explainers is complex and driven by several interconnected factors and objectives ( [24,23]). In general, this type of explainers can be unstable and lead to varying surrogate coefficients and, in consequence, diverse explanations ( [1,32,33]).…”

Section: Related Workmentioning

confidence: 99%

Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Schulz¹,

Poyiadzi²,

Santos-Rodríguez³

2022

nldl

View full text Add to dashboard Cite

Explainability of black-box machine learning models is crucial, in particular when deployed in critical applications such as medicine or autonomous cars. Existing approaches produce explanations for the predictions of models, however, how to assess the quality and reliability of such explanations remains an open question. In this paper we take a step further in order to provide the practitioner with tools to judge the trustworthiness of an explanation. To this end, we produce estimates of the uncertainty of a given explanation by measuring the ordinal consensus amongst a set of diverse bootstrapped surrogate explainers. While we encourage diversity by using ensemble techniques, we propose and analyse metrics to aggregate the information contained within the set of explainers through a rating scheme. We empirically illustrate the properties of this approach through experiments on state-of-the-art Convolutional Neural Network ensembles. Furthermore, through tailored visualisations, we show specific examples of situations where uncertainty estimates offer concrete actionable insights to the user beyond those arising from standard surrogate explainers.

show abstract

Section: Related Workmentioning

confidence: 99%

Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Schulz¹,

Poyiadzi²,

Santos-Rodríguez³

2022

nldl

View full text Add to dashboard Cite

show abstract

“…The process of deriving surrogate explainers is complex and driven by several interconnected factors and objectives ( [22,21]). In general, this type of explainers can be unstable and lead to varying surrogate coefficients and, in consequence, diverse explanations ( [1,30,31]).…”

Section: Related Workmentioning

confidence: 99%

Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Schulz¹,

Poyiadzi²,

Santos-Rodríguez³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Explainability of black-box machine learning models is crucial, in particular when deployed in critical applications such as medicine or autonomous cars. Existing approaches produce explanations for the predictions of models, however, how to assess the quality and reliability of such explanations remains an open question. In this paper we take a step further in order to provide the practitioner with tools to judge the trustworthiness of an explanation. To this end, we produce estimates of the uncertainty of a given explanation by measuring the ordinal consensus amongst a set of diverse bootstrapped surrogate explainers. While we encourage diversity by using ensemble techniques, we propose and analyse metrics to aggregate the information contained within the set of explainers through a rating scheme. We empirically illustrate the properties of this approach through experiments on state-of-theart Convolutional Neural Network ensembles. Furthermore, through tailored visualisations, we show specific examples of situations where uncertainty estimates offer concrete actionable insights to the user beyond those arising from standard surrogate explainers.

show abstract

“…We focus on both the groups of interpretation methods, including local as well as global interpretations. Among the local explanation methods, the well-known method is LIME (Local Interpretable Model-Agnostic Explanation) [11], which belongs to methods based on constructing a surrogate model [12] approximating the black-box model at a point. The surrogate model in LIME is the linear regression whose coefficients can be interpreted as the feature importance measures.…”

Section: Introductionmentioning

confidence: 99%

Attention-like feature explanation for tabular data

Konstantinov¹,

Utkin²

2021

Preprint

View full text Add to dashboard Cite

A new method for local and global explanation of the machine learning black-box model predictions by tabular data is proposed. It is implemented as a system called AFEX (Attention-like Feature EXplanation) and consisting of two main parts. The first part is a set of the one-feature neural subnetworks which aim to get a specific representation for every feature in the form of a basis of shape functions. The subnetworks use shortcut connections with trainable parameters to improve the network performance. The second part of AFEX produces shape functions of features as the weighted sum of the basis shape functions where weights are computed by using an attention-like mechanism. AFEX identifies pairwise interactions between features based on pairwise multiplications of shape functions corresponding to different features. A modification of AFEX with incorporating an additional surrogate model which approximates the black-box model is proposed. AFEX is trained end-to-end on a whole dataset only once such that it does not require to train neural networks again in the explanation stage. Numerical experiments with synthetic and real data illustrate AFEX.

show abstract

Understanding surrogate explanations: the interplay between complexity, fidelity and coverage

Cited by 3 publications

References 7 publications

Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Attention-like feature explanation for tabular data

Contact Info

Product

Resources

About