Application of Instruction-Based Behavior Explanation to a Reinforcement Learning Agent with Changing Policy

Fukuchi, Yosuke; Osawa, Masahiko; Yamakawa, Hiroshi; Imai, Michita

doi:10.1007/978-3-319-70087-8_11

Cited by 3 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fukuchi et al ( 2017a ) described the Instruction-based Behavior Explanation (IBE) approach which allows an agent to announce their future behavior. To accomplish this, the agent leveraged Interactive RL where experts provide instructions in real-time to beginner agents.…”

Section: Discussionmentioning

confidence: 99%

“…While a policy summary doesn't explain an individual action, it can help provide context for why an action was taken, and more broadly why an agent makes the overall set of actions it makes. Fukuchi et al (2017a) described the Instruction-based Behavior Explanation (IBE) approach which allows an agent to announce their future behavior. To accomplish this, the agent leveraged Interactive RL where experts provide instructions in real-time to beginner agents.…”

Section: Policy Summarizationmentioning

confidence: 99%

See 1 more Smart Citation

Explainable AI and Reinforcement Learning—A Systematic Review of Current Approaches and Trends

Wells

Bednarz

2021

Front. Artif. Intell.

101

View full text Add to dashboard Cite

Research into Explainable Artificial Intelligence (XAI) has been increasing in recent years as a response to the need for increased transparency and trust in AI. This is particularly important as AI is used in sensitive domains with societal, ethical, and safety implications. Work in XAI has primarily focused on Machine Learning (ML) for classification, decision, or action, with detailed systematic reviews already undertaken. This review looks to explore current approaches and limitations for XAI in the area of Reinforcement Learning (RL). From 520 search results, 25 studies (including 5 snowball sampled) are reviewed, highlighting visualization, query-based explanations, policy summarization, human-in-the-loop collaboration, and verification as trends in this area. Limitations in the studies are presented, particularly a lack of user studies, and the prevalence of toy-examples and difficulties providing understandable explanations. Areas for future study are identified, including immersive visualization, and symbolic representation.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Policy Summarizationmentioning

confidence: 99%

Explainable AI and Reinforcement Learning—A Systematic Review of Current Approaches and Trends

Wells

Bednarz

2021

Front. Artif. Intell.

101

View full text Add to dashboard Cite

show abstract

Explainable reinforcement learning (XRL): a systematic literature review and taxonomy

Bekkemoen

2023

Mach Learn

View full text Add to dashboard Cite

In recent years, reinforcement learning (RL) systems have shown impressive performance and remarkable achievements. Many achievements can be attributed to combining RL with deep learning. However, those systems lack explainability, which refers to our understanding of the system’s decision-making process. In response to this challenge, the new explainable RL (XRL) field has emerged and grown rapidly to help us understand RL systems. This systematic literature review aims to give a unified view of the field by reviewing ten existing XRL literature reviews and 189 XRL studies from the past five years. Furthermore, we seek to organize these studies into a new taxonomy, discuss each area in detail, and draw connections between methods and stakeholder questions (e.g., “how can I get the agent to do _?”). Finally, we look at the research trends in XRL, recommend XRL methods, and present some exciting research directions for future research. We hope stakeholders, such as RL researchers and practitioners, will utilize this literature review as a comprehensive resource to overview existing state-of-the-art XRL methods. Additionally, we strive to help find research gaps and quickly identify methods that answer stakeholder questions.

show abstract

Explaining Intelligent Agent’s Future Motion on Basis of Vocabulary Learning With Human Goal Inference

et al. 2022

Self Cite

View full text Add to dashboard Cite

Intelligent agents (IAs) that use machine learning for decision-making often lack the explainability about what they are going to do, which makes human-IA collaboration challenging. However, previous methods of explaining IA behavior require IA developers to predefine vocabulary that expresses motion, which is problematic as IA decision-making becomes complex. This paper proposes Manifestor, a method for explaining an IA's future motion with autonomous vocabulary learning. With Manifestor, an IA can learn vocabulary from a person's instructions about how the IA should act. A notable contribution of this paper is that we formalized the communication gap between a person and IA in the vocabulary-learning phase, that is, the IA's goal may be different from what the person wants the IA to achieve, and the IA needs to infer the latter to judge whether a motion matches that person's instruction. We evaluated Manifestor by investigating whether people can accurately predict an IA's future motion with explanations generated with Manifestor. We compared Manifestor's vocabulary with that from optimal acquired in a situation in which the communication-gap problem did not exist and that from ablation, which was learned with a false assumption that an IA and person shared a goal. The experimental results revealed that vocabulary learned with Manifestor improved people's prediction accuracy as much as with optimal, while ablation failed, suggesting that Manifestor can enable an IA to properly learn vocabulary from people's instructions even if a communication gap exists.INDEX TERMS Explainable AI, Human-agent interaction, Intelligent agent, Deep reinforcement learning.

show abstract

Application of Instruction-Based Behavior Explanation to a Reinforcement Learning Agent with Changing Policy

Cited by 3 publications

References 7 publications

Explainable AI and Reinforcement Learning—A Systematic Review of Current Approaches and Trends

Explainable AI and Reinforcement Learning—A Systematic Review of Current Approaches and Trends

Explainable reinforcement learning (XRL): a systematic literature review and taxonomy

Explaining Intelligent Agent’s Future Motion on Basis of Vocabulary Learning With Human Goal Inference

Contact Info

Product

Resources

About