Four principles of explainable artificial intelligence

Abstract

We introduce four principles for explainable artificial intelligence (AI) that comprise fundamental properties for explainable AI systems. We propose that explainable AI systems deliver accompanying evidence or reasons for outcomes and processes; provide explanations that are understandable to individual users; provide explanations that correctly reflect the system's process for generating the output; and that a system only operates under conditions for which it was designed and when it reaches sufficient confidence in its output. We have termed these four principles as explanation, meaningful, explanation accuracy, and knowledge limits, respectively. Through significant stakeholder engagement, these four principles were developed to encompass the multidisciplinary nature of explainable AI, including the fields of computer science, engineering, and psychology. Because one-sizefits-all explanations do not exist, different users will require different types of explanations. We present five categories of explanation and summarize theories of explainable AI. We give an overview of the algorithms in the field that cover the major classes of explainable algorithms. As a baseline comparison, we assess how well explanations provided by people follow our four principles. This assessment provides insights to the challenges of designing explainable AI systems.

Fairness and Transparency in Recommendation: The Users’ Perspective

“…One of our key areas of interest in this study is the question of how to design for transparency in the context of fairness-aware algorithms. Transparency has become an essential feature of algorithmic design, especially in application areas that are socially sensitive due to (1) legal requirements for transparency such as the GDPR [24,41]; (2) the need to build users' trust in a system [11,29,38]; (3) the need for error detection to help mitigate bias and discrimination in a system [23,37,48,49]; (4) helping further public adoption of new technologies [1,3,17,25,30]; and/or (5) creating an environment of accountability for the platforms that host the algorithms [1,15]. We believe that transparency only increases in importance for fairness-aware systems [32].…”

Multisource AI scorecard table analysis of AMIGO

“…A recent taxonomy follows these definitions to include similar approaches of principles for explainable AI from the National Institute of Standards (NIST) [5]. The NIST principles of xAI focus on the user and are derived the Defense Advance Research Project Agency (DARPA) program [6].…”

“…In assessing the connection between the user and the man-machine interaction, Philips et al [5] approached explainable AI (xAI) in a similar fashion with four principles for xAI. The explanations:…”

Holistic Explainability Requirements for End-to-End Machine Learning in IoT Cloud Systems

“…Explainability in ML is an important subject intensively studied in recent years. There is a wide range of topics that has been discussed, including explainability for multiple stakeholders [5], requirements for explainability [6], [11], and systems for achieving explainability in ML models [10], [7]. Table I summarizes the main related work in the explainability requirements.…”