Explainability fact sheets

Abstract

Explanations in Machine Learning come in many forms, but a consensus regarding their desired properties is yet to emerge. In this paper we introduce a taxonomy and a set of descriptors that can be used to characterise and systematically assess explainable systems along five key dimensions: functional, operational, usability, safety and validation. In order to design a comprehensive and representative taxonomy and associated descriptors we surveyed the eXplainable Artificial Intelligence literature, extracting the criteria and desiderata that other authors have proposed or implicitly used in their research. The survey includes papers introducing new explainability algorithms to see what criteria are used to guide their development and how these algorithms are evaluated, as well as papers proposing such criteria from both computer science and social science perspectives. This novel framework allows to systematically compare and contrast explainability approaches, not just to better understand their capabilities but also to identify discrepancies between their theoretical qualities and properties of their implementations. We developed an operationalisation of the framework in the form of Explainability Fact Sheets, which enable researchers and practitioners alike to quickly grasp capabilities and limitations of a particular explainable method. When used as a Work Sheet, our taxonomy can guide the development of new explainability approaches by aiding in their critical evaluation along the five proposed dimensions.

What do we want from Explainable Artificial Intelligence (XAI)? – A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research

“…Gain scientific insights from the system User [5,7,12,14,35,37,44,47,60,63,66,67,70,87,97,100,106] Security Assess and increase a system's security [13,16,25,26,29,33,34,37,39,44,46,49,55,57,62,63,67,74,75,78,80,92,94,107,108] [ 28,31,43,50] Trust Calibrate appropriate trust in the system User, Deployer…”

“…This degree of understanding can, then, be augmented (e.g., with explanatory information generated from explainability approaches). With a higher degree of understanding (and, consequently, a more detailed and accurate mental model of a system), a stakeholder might understand what kind of training data underlie a given system, what kind of algorithm is used for a given system, or what kind of output data a system produces [17,63,159].…”

“…Additionally, it can happen that understanding contributes to the satisfaction of a single desideratum of a stakeholder to a greater extent, while the satisfaction of other desiderata for the same stakeholder suffers [63].…”

“…In other words, it should not be possible for particular users to manipulate a system in such a way that they can systematically evoke beneficial outputs for them. However, more transparency caused by a higher degree of understanding may enable some people to exploit the system [63]. In a personnel selection test, for example, a better understanding of the selection system may enable participants to game the test, preventing its proper use (i.e., selecting suitable applicants).…”

“…Additionally, these patterns may help stakeholders to tell apart correct but unexpected behavior from malfunctions in order to to debug the system. In general, all of this can reduce people's uncertainty concerning a system (e.g., uncertainty concerning how to behave towards it, how to react to it, what to think of it, whether to recommend it, or whether to disseminate it; [63]). Overall, explanatory information should lead to a better understanding, which should, in turn, positively affect the satisfaction of stakeholders' desiderata.…”