Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach

Abstract

Explainability of black-box machine learning models is crucial, in particular when deployed in critical applications such as medicine or autonomous cars. Existing approaches produce explanations for the predictions of models, however, how to assess the quality and reliability of such explanations remains an open question. In this paper we take a step further in order to provide the practitioner with tools to judge the trustworthiness of an explanation. To this end, we produce estimates of the uncertainty of a given explanation by measuring the ordinal consensus amongst a set of diverse bootstrapped surrogate explainers. While we encourage diversity by using ensemble techniques, we propose and analyse metrics to aggregate the information contained within the set of explainers through a rating scheme. We empirically illustrate the properties of this approach through experiments on state-of-theart Convolutional Neural Network ensembles. Furthermore, through tailored visualisations, we show specific examples of situations where uncertainty estimates offer concrete actionable insights to the user beyond those arising from standard surrogate explainers.

Local Model Explanations and Uncertainty Without Model Access

“…Related Work While our work is unique in that we do not assume access to the underlying ML model, our approach builds upon previous methods that attempt to quantify the uncertainty of explanations. These include methods that apply standard sampling error techniques to explainability [30] as well as non-parametric approaches [27,35]. While [27] also employs bootstrap methods, our approach is different in that we focus on gradient estimation of the ML model instead of rank orders of feature importance.…”

“…These include methods that apply standard sampling error techniques to explainability [30] as well as non-parametric approaches [27,35]. While [27] also employs bootstrap methods, our approach is different in that we focus on gradient estimation of the ML model instead of rank orders of feature importance. Additionally, [35] focuses on quantifying the uncertainty in explanations of convolutional neural networks, while our method is model-agnostic.…”