Levels of explainable artificial intelligence for human-aligned conversational explanations

Dazeley, Richard; Vamplew, Peter; Foale, Cameron; Young, Craig; Aryal, Sunil; Cruz, Francisco

doi:10.1016/j.artint.2021.103525

Cited by 68 publications

(69 citation statements)

References 203 publications

(280 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, it would not be useful to have an algorithm that recommended movies we do not like, or, also, it would be serious if a medical system made a faulty negative diagnosis during an evaluation of a patient with cancer. In this context, the role of explainable artificial intelligence is an important role since it consists of tools, techniques, and algorithms that provide the agent with the ability to explain its action to the human intuitively [6,7].…”

Section: Introductionmentioning

confidence: 99%

Analysis of Explainable Goal-Driven Reinforcement Learning in a Continuous Simulated Environment

et al. 2022

Self Cite

View full text Add to dashboard Cite

Currently, artificial intelligence is in an important period of growth. Due to the technology boom, it is now possible to solve problems that could not be resolved previously. For example, through goal-driven learning, it is possible that intelligent machines or agents may be able to perform tasks without human intervention. However, this also leads to the problem of understanding the agent’s decision making. Therefore, explainable goal-driven learning attempts to eliminate this gap. This work focuses on the adaptability of two explainability methods in continuous environments. The methods based on learning and introspection proposed a probability value for success to explain the agent’s behavior. These had already been tested in discrete environments. The continuous environment used in this study is the car-racing problem. This is a simulated car racing game that forms part of the Python Open AI Gym Library. The agents in this environment were trained with the Deep Q-Network algorithm, and in parallel the explainability methods were implemented. This research included a proposal for carrying out the adaptation and implementation of these methods in continuous states. The adaptation of the learning method produced major changes, implemented through an artificial neural network. The obtained probabilities of both methods were consistent throughout the experiments. The probability result was greater in the learning method. In terms of computational resources, the introspection method was slightly better than its counterpart.

show abstract

Section: Introductionmentioning

confidence: 99%

Analysis of Explainable Goal-Driven Reinforcement Learning in a Continuous Simulated Environment

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, for AI to succeed, it must provide trusted and socially acceptable systems and, therefore, should be modelled on philosophical, psychological and cognitive science models of human explanation. Dazeley et al [58] identified a set of levels of explainability that are adapted from Animal Cognitive Ethology's levels of intentionality [91,45], combined with human social contexts [57]. After reviewing the literature across these levels, Dazeley et al [58] shows that the majority of XAI research is focused on the lowest level of Zero-order explanations.…”

Section: Introductionmentioning

confidence: 99%

“…Dazeley et al [58] identified a set of levels of explainability that are adapted from Animal Cognitive Ethology's levels of intentionality [91,45], combined with human social contexts [57]. After reviewing the literature across these levels, Dazeley et al [58] shows that the majority of XAI research is focused on the lowest level of Zero-order explanations. Furthermore, that a fully Broad-XAI system requires the full range of XAI levels to provide an integrated conversational explanation.…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, that a fully Broad-XAI system requires the full range of XAI levels to provide an integrated conversational explanation. While Dazeley et al [58] argues that a mix of technologies can be used to develop broad-XAI, RL approaches were identified as one approach for providing a backbone supporting explanations across multiple levels of explainability.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey

Dazeley¹,

Vamplew²,

Cruz³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Broad Explainable Artificial Intelligence (Broad-XAI) moves away from interpreting individual decisions based on a single datum and aims to provide integrated explanations from multiple machine learning algorithms into a coherent explanation of an agent's behaviour that is aligned to the communication needs of the explainee. Reinforcement Learning (RL) methods, we propose, provide a potential backbone for the cognitive model required for the development of Broad-XAI. RL represents a suite of approaches that have had increasing success in solving a range of sequential decision-making problems. However, these algorithms all operate as black-box problem solvers, where they obfuscate their decision-making policy through a complex array of values and functions. EXplainable RL (XRL) is relatively recent field of research that aims to develop techniques to extract concepts from the agent's: perception of the environment; intrinsic/extrinsic motivations/beliefs; Q-values, goals and objectives. This paper aims to introduce a conceptual framework, called the Causal XRL Framework (CXF), that unifies the current XRL research and uses RL as a backbone to the development of Broad-XAI. Additionally, we recognise that RL methods have the ability to incorporate a range of technologies to allow agents to adapt to their environment. CXF is designed for the incorporation of many standard RL extensions and integrated with external ontologies and communication facilities so that the agent can answer questions that explain outcomes and justify its decisions. This paper aims to: establish XRL as a distinct branch of Explainable Artificial Intelligence (XAI); introduce a conceptual framework for XRL; review existing approaches explaining agent behaviour; and identify opportunities for future research. Finally, this paper will discuss how additional information can be extracted and ultimately integrated into models of communication, facilitating the development of Broad-XAI.

show abstract

“…They often fail to understand the meaning of a particular observation, which may be obvious to a human observer, thus missing the opportunity for the agent to learn [31]. Vice versa, agents that provide explanations for their actions are essential for humans to learn from experiences [32]. Generating an explanation requires access to some sort of internal representation about the task, the team, and the situational context.…”

mentioning

confidence: 99%

“I’m Afraid I Can’t Do That, Dave”; Getting to Know Your Buddies in a Human–Agent Team

Schadd¹,

Schoonderwoerd²,

Bosch³

et al. 2022

Systems

View full text Add to dashboard Cite

The rapid progress in artificial intelligence enables technology to more and more become a partner of humans in a team, rather than being a tool. Even more than in human teams, partners of human–agent teams have different strengths and weaknesses, and they must acknowledge and utilize their respective capabilities. Coordinated team collaboration can be accomplished by smartly designing the interactions within human–agent teams. Such designs are called Team Design Patterns (TDPs). We investigated the effects of a specific TDP on proactive task reassignment. This TDP supports team members to dynamically allocate tasks by utilizing their knowledge about the task demands and about the capabilities of team members. In a pilot study, agent–agent teams were used to study the effectiveness of proactive task reassignment. Results showed that this TDP improves a team’s performance, provided that partners have accurate knowledge representations of each member’s skill level. The main study of this paper addresses the effects of task reassignments in a human–agent team. It was hypothesized that when agents provide explanations when issuing and responding to task reassignment requests, this will enhance the quality of the human’s mental model. Results confirmed that participants developed more accurate mental models when agent-partners provide explanations. This did not result in a higher performance of the human–agent team, however. The study contributes to our understanding of designing effective collaboration in human–agent teams.

show abstract

Levels of explainable artificial intelligence for human-aligned conversational explanations

Cited by 68 publications

References 203 publications

Analysis of Explainable Goal-Driven Reinforcement Learning in a Continuous Simulated Environment

Analysis of Explainable Goal-Driven Reinforcement Learning in a Continuous Simulated Environment

Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Survey

“I’m Afraid I Can’t Do That, Dave”; Getting to Know Your Buddies in a Human–Agent Team

Contact Info

Product

Resources

About