Aspects of Intelligent Systems Explanation

Darlington, Keith

doi:10.13189/ujca.2013.010204

Cited by 20 publications

(20 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the deployment of expert systems, it became clear that explanations [Dar13] are crucial for human users' trust [SM93,WPH16] and acceptability of these AI systems [YJ95]. We formulated two questions regarding the transparency of the decision provided by artificial agents.…”

Section: Analysis As To Whether Justifications Are Necessarymentioning

confidence: 99%

Randomized Classifiers vs Human Decision-Makers: Trustworthy AI May Have to Act Randomly and Society Seems to Accept This

Erdélyi¹,

Erdélyi²,

Estivill‐Castro³

2021

Preprint

View full text Add to dashboard Cite

As artificial intelligence (AI) systems are increasingly involved in decisions affecting our lives, ensuring that automated decision-making is fair and ethical has become a top priority. Intuitively, we feel that akin to human decisions, judgments of artificial agents should necessarily be grounded in some moral principles. Yet a decision-maker (whether human or artificial) can only make truly ethical (based on any ethical theory) and fair (according to any notion of fairness) decisions if full information on all the relevant factors on which the decision is based are available at the time of decision-making. This raises two problems: (1) In settings, where we rely on AI systems that are using classifiers obtained with supervised learning, some induction/generalization is present and some relevant attributes may not be present even during learning. (2) Modeling such decisions as games reveals that any-however ethical-pure strategy is inevitably susceptible to exploitation. Moreover, in many games, a Nash Equilibrium can only be obtained by using mixed strategies, i.e., to achieve mathematically optimal outcomes, decisions must be randomized. In this paper, we argue that in supervised learning settings, there exist random classifiers that perform at least as well as deterministic classifiers, and may hence be the optimal choice in many circumstances. We support our theoretical results with an empirical study indicating a positive societal attitude towards randomized artificial decision-makers, and discuss some policy and implementation issues related to the use of random classifiers that relate to and are relevant for current AI policy and standardization initiatives.

show abstract

Section: Analysis As To Whether Justifications Are Necessarymentioning

confidence: 99%

Randomized Classifiers vs Human Decision-Makers: Trustworthy AI May Have to Act Randomly and Society Seems to Accept This

Erdélyi¹,

Erdélyi²,

Estivill‐Castro³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The importance of generating explanations or justifications of decisions made by an AI system has been emphasized and widely explored in numerous works over the past decades [10], [25], [26], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83], [84], [85]. Most prior work in explaining CNN's predictions has focused on generating explanations using feature visualization and attribution.…”

Section: Related Workmentioning

confidence: 99%

CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models

Akula¹,

Wang²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose CX-ToM, short for counterfactual explanations with theory-of mind, a new explainable AI (XAI) framework for explaining decisions made by a deep convolutional neural network (CNN). In contrast to the current methods in XAI that generate explanations as a single shot response, we pose explanation as an iterative communication process, i.e. dialog, between the machine and human user. More concretely, our CX-ToM framework generates sequence of explanations in a dialog by mediating the differences between the minds of machine and human user. To do this, we use Theory of Mind (ToM) which helps us in explicitly modeling human's intention, machine's mind as inferred by the human as well as human's mind as inferred by the machine. Moreover, most state-of-the-art XAI frameworks provide attention (or heat map) based explanations. In our work, we show that these attention based explanations are not sufficient for increasing human trust in the underlying CNN model. In CX-ToM, we instead use counterfactual explanations called fault-lines which we define as follows: given an input image I for which a CNN classification model M predicts class c pred , a fault-line identifies the minimal semantic-level features (e.g., stripes on zebra, pointed ears of dog), referred to as explainable concepts, that need to be added to or deleted from I in order to alter the classification category of I by M to another specified class c alt . We argue that, due to the iterative, conceptual and counterfactual nature of CX-ToM explanations, our framework is practical and more natural for both expert and non-expert users to understand the internal workings of complex deep learning models. Extensive quantitative and qualitative experiments verify our hypotheses, demonstrating that our CX-ToM significantly outperforms the state-of-the-art explainable AI models.

show abstract

“…Primarily, explainability can help developers to detect errors and, thus, can lead to increased system debugability, facilitating system safety and robustness [10]. Further benefits are verification and validation, which become easier through explainability [22]. Overall, the technical perspective is a foundation for all other perspectives on explainability because without the technical perspective, requirements from other perspective cannot be met.…”

Section: A Technical Perspectivementioning

confidence: 99%

“…However, if it is a decision based solely on automated processing that produces legal effects concerning the data subject or significantly affects them, it is only permissible if the requirements of Art. 22 (2) GDPR are fulfilled. This is the case if the decision is necessary for a contract between the data subject and the controller, if it is required by law, or if it is based on the data subject's explicit consent.…”

Section: Legal Perspective 1) Legal Dimensionsmentioning

confidence: 99%

Explainability Auditing for Intelligent Systems: A Rationale for Multi-Disciplinary Perspectives

Langer¹,

Baum²,

Hartmann³

et al. 2021

Preprint

View full text Add to dashboard Cite

National and international guidelines for trustworthy artificial intelligence (AI) consider explainability to be a central facet of trustworthy systems. This paper outlines a multidisciplinary rationale for explainability auditing. Specifically, we propose that explainability auditing can ensure the quality of explainability of systems in applied contexts and can be the basis for certification as a means to communicate whether systems meet certain explainability standards and requirements. Moreover, we emphasize that explainability auditing needs to take a multidisciplinary perspective, and we provide an overview of four perspectives (technical, psychological, ethical, legal) and their respective benefits with respect to explainability auditing.

show abstract

Aspects of Intelligent Systems Explanation

Cited by 20 publications

References 34 publications

Randomized Classifiers vs Human Decision-Makers: Trustworthy AI May Have to Act Randomly and Society Seems to Accept This

Randomized Classifiers vs Human Decision-Makers: Trustworthy AI May Have to Act Randomly and Society Seems to Accept This

CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models

Explainability Auditing for Intelligent Systems: A Rationale for Multi-Disciplinary Perspectives

Contact Info

Product

Resources

About