One Explanation Does Not Fit All

Abstract

The need for transparency of predictive systems based on Machine Learning algorithms arises as a consequence of their ever-increasing proliferation in the industry. Whenever black-box algorithmic predictions influence human affairs, the inner workings of these algorithms should be scrutinised and their decisions explained to the relevant stakeholders, including the system engineers, the system's operators and the individuals whose case is being decided. While a variety of interpretability and explainability methods is available, none of them is a panacea that can satisfy all diverse expectations and competing objectives that might be required by the parties involved. We address this challenge in this paper by discussing the promises of Interactive Machine Learning for improved transparency of black-box systems using the example of contrastive explanations-a state-of-the-art approach to Interpretable Machine Learning. Specifically, we show how to personalise counterfactual explanations by interactively adjusting their conditional statements and extract additional explanations by asking follow-up "What if?" questions. Our experience in building, deploying and presenting this type of system allowed us to list desired properties as well as potential limitations, which can be used to guide the development of interactive explainers. While customising the medium of interaction, i.e., the user interface comprising of various communication channels, may give an impression of personalisation, we argue that adjusting the explanation itself and its content is more important. To this end, properties such as breadth, scope, context, purpose and target of the explanation have to be considered, in addition to explicitly informing the explainee about its limitations and caveats. Furthermore, we discuss the challenges of mirroring the explainee's mental model, which is the main building block of intelligible human-machine interactions. We also deliberate on the risks of allowing the explainee to freely manipulate the explanations and thereby extracting information about the underlying predictive model, which might be leveraged by malicious actors to steal or game the model. Finally, building an end-to-end interactive explainability system is a challenging engineering task; unless the main goal is its deployment, we recommend "Wizard of Oz" studies as a proxy for testing and evaluating standalone interactive explainability algorithms.

Explanation as a Social Practice: Toward a Conceptual Framework for the Social Design of AI Systems

“…Following the call to extend XAI to social and interactive approaches, novel developments in XAI acknowledge the diversity in explainees in terms of their expectations, interests, and needs as a way to personalize explanations [14,15,11,16,7]. These approaches characterize the explainee in terms of a number of variables that specify the person's characteristics…”

“…To sum up the above-mentioned limitations, current research in XAI lacks a conceptual framework that would account for both explaining as a bidirectional social process and the dynamics of understanding in everyday explanations. A conceptual basis that allows one to assess and describe the process of explaining could enhance the design of AI systems tremendously [7] by orienting them toward the production of socially relevant explanations that cover not only specific points of interest in the addressee, but also the general dynamics of the process taking place on different levels. A conceptual basis is necessary, because in a recent review, Anjomoshoae et al [6, p. 1082] revealed that 39% of research concerned with explainable and intelligent agents "did not rely on any theoretical background related to generating explanations"thereby strongly suggesting that current theories might be lagging behind what designers of AI systems already recognize as being more appropriate.…”

“…Our article takes this conclusion as a starting point. Following [7], we argue that one important source from which XAI can tap diverse expectations is the interactive process of explaining. In this process, the receiver of an explanation does not just play a passive role of providing a set of properties according to which an explanation needs to be "personalized."…”

“…In this respect, we argue that the explainee should be characterized by a complex and dynamic variable that captures not only personal traits but also the explainee's continuously changing level of understanding. In other words, the extension of XAI research to social and interactive approaches (as suggested by [10] and [7]) should not limit itself to characterizing an explainee as an individuum with preferences, personal characteristics, intentions, etc. Instead, it is necessary to account for the explainee seeking to understand the explanandum.…”

Investigating the Impact of Explanation on Repairing Trust in Ai Diagnostic Systems for Re-Diagnosis

“…Google's people + AI Guidebook has described the best practices for designing human centered AI products and acknowledging the importance of interaction and explainability 17 . There are also other AI systems that personalize explanation in interactive environment (Akula et al, 2019;Schneider and Handali, 2019;Sokol and Flach, 2020), but the necessity of tailoring explanation to the need of different patients has not been discussed in the past literature. In my interview study, physicians often stated that they develop their explanation considering the patient's emotional, cultural, or socioeconomic status, they also have to keep in mind the intellectual level of the patients.…”