A decision-theoretic approach for model interpretability in Bayesian framework

Abstract

A salient approach to interpretable machine learning is to restrict modeling to simple models. In the Bayesian framework, this can be pursued by restricting the model structure and prior to favor interpretable models. Fundamentally, however, interpretability is about users’ preferences, not the data generation mechanism; it is more natural to formulate interpretability as a utility function. In this work, we propose an interpretability utility, which explicates the trade-off between explanation fidelity and interpretability in the Bayesian framework. The method consists of two steps. First, a reference model, possibly a black-box Bayesian predictive model which does not compromise accuracy, is fitted to the training data. Second, a proxy model from an interpretable model family that best mimics the predictive behaviour of the reference model is found by optimizing the interpretability utility function. The approach is model agnostic—neither the interpretable model nor the reference model are restricted to a certain class of models—and the optimization problem can be solved using standard tools. Through experiments on real-word data sets, using decision trees as interpretable models and Bayesian additive regression models as reference models, we show that for the same level of interpretability, our approach generates more accurate models than the alternative of restricting the prior. We also propose a systematic way to measure stability of interpretabile models constructed by different interpretability approaches and show that our proposed approach generates more stable models.

Machine Learning Approach to Delineate the Impact of Material Properties on Solar Cell Device Physics

“…However as decision tree predictions are piecewise constant approximations, rather than continuous predictions, it is challenging to extrapolate them. 80 This indicates that a numerical value higher than the maximum value of a given output cannot be predicted. The output value beyond the maximum point loses any physical meaning.…”

“…This behavior indicates that EA should be optimized to be a value near 4.25 eV in practice. However as decision tree predictions are piecewise constant approximations, rather than continuous predictions, it is challenging to extrapolate them . This indicates that a numerical value higher than the maximum value of a given output cannot be predicted.…”

Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties

“…Nevertheless, we demonstrate empirically that our approach is able to match or exceed the performance of existing methods, with respect to the realism of the CEs generated. In future work, methodological developments could be explored by adapting the proposed method to work for black-box models (Afrabandpey et al, 2020).…”

Projection Predictive Inference for Generalized Linear and Additive Multilevel Models

“…Reference models have been used before for tasks other than variable selection, as in Afrabandpey et al (2020), where the authors constrain the projection of a complex neural network to be interpretable (e.g., projecting onto decision trees). Closer to variable selection and related to our approach, Piironen and Vehtari (2016) use projection predictive inference and impose further constraints on the projection of a GP reference model to perform variable selection, given the identifiability issue of the direct projection.…”