Individual Explanations in Machine Learning Models: A Survey for Practitioners

Abstract

In recent years, the use of sophisticated statistical models that influence decisions in domains of high societal relevance is on the rise. Although these models can often bring substantial improvements in the accuracy and efficiency of organizations, many governments, institutions, and companies are reluctant to their adoption as their output is often difficult to explain in human-interpretable ways. Hence, these models are often regarded as black-boxes, in the sense that their internal mechanisms can be opaque to human audit. In real-world applications, particularly in domains where decisions can have a sensitive impact-e.g., criminal justice, estimating credit scores, insurance risk, health risks, etc.-model interpretability is desired. Recently, the academic literature has proposed a substantial amount of methods for providing interpretable explanations to machine learning models. This survey reviews the most relevant and novel methods that form the state-of-the-art for addressing the particular problem of explaining individual instances in machine learning. It seeks to provide a succinct review that can guide data science and machine learning practitioners in the search for appropriate methods to their problem domain.

Intraoperative Hypotension Prediction Based on Features Automatically Generated Within an Interpretable Deep Learning Model

“…Statistical models may have advantages over common interpretable methods such as SHAP because they provide generalizable interpretation. Specifically, the SHAP values change according to the distribution of the given input data [18], making it difficult to generalize the interpretation. However, since pvalues and odd ratios are constant regardless of the given input data, logistic regression can provide a generalizable interpretation [43].…”

“…The last limitation is the lack of flexibility of the post-hoc interpretation, which is caused by the dependence of modelspecific interpretation on the model structure [18]. Specifically, because the ABP trend shapes were generated based on 30 s of ABP in this study, the association between the hypotension development and ABP trends at different time lengths such as 40 s is not interpretable.…”

“…Modelspecific methods focus on constructing a transparent mechanism that allows intrinsic interpretation of the model itself. Examples include variable importance computed from boosting or bagging machine learning algorithms and feature maps extracted from certain layers or weights in neural networks [17], [18]. In contrast, model-agnostic methods are applied independently from the model by approximating the relationship between input and output data.…”

“…In contrast, model-agnostic methods are applied independently from the model by approximating the relationship between input and output data. Shapley additive explanation (SHAP) and local interpretable model-agnostic explanations (LIME) are representative examples [17], [18].…”

Intraoperative Hypotension Prediction Based on Features Automatically Generated Within an Interpretable Deep Learning Model

“…Modelspecific methods focus on constructing a transparent mechanism that allows intrinsic interpretation of the model itself. Examples include variable importance computed from boosting or bagging machine learning algorithms and feature maps extracted from certain layers or weights in a neural network [19], [20]. By contrast, model-agnostic methods are applied independently of the model by approximating the relationship between the input and output data.…”