A Survey of Explainable Reinforcement Learning

Milani, Stephanie; Topin, Nicholay; Veloso, Manuela; Fang, F.

doi:10.48550/arxiv.2202.08434

Cited by 17 publications

(24 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second one relates to the models not being built off labelled training data (which would simplify explainability). Further inspiration could be taken from relevant survey papers and implementations [10,49,74,75,80,82,91,95,96,109,116]. Failure to address this challenge will lead to the automated blue agent not being certified by industrial employees within networked systems since the trust towards the agent will be low.…”

Section: Explainable Rl (A24)mentioning

confidence: 99%

Automated Cyber Defence: A Review

Vyas¹,

Hannay²,

Bolton³

et al. 2023

Preprint

View full text Add to dashboard Cite

Within recent times, cybercriminals have curated a variety of organised and resolute cyber attacks within a range of cyber systems, leading to consequential ramifications to private and governmental institutions. Current security-based automation and orchestrations focus on automating fixed purpose and hard-coded solutions, which are easily surpassed by modern-day cyber attacks. Research within Automated Cyber Defence will allow the development and enabling intelligence response by autonomously defending networked systems through sequential decision-making agents. This article comprehensively elaborates the developments within Automated Cyber Defence through a requirement analysis divided into two sub-areas, namely, automated defence and attack agents and Autonomous Cyber Operation (ACO) Gyms. The requirement analysis allows the comparison of automated agents and highlights the importance of ACO Gyms for their continual development. The requirement analysis is also used to critique ACO Gyms with an overall aim to develop them for deploying automated agents within real-world networked systems. Relevant future challenges were addressed from the overall analysis to accelerate development within the area of Automated Cyber Defence.

show abstract

Section: Explainable Rl (A24)mentioning

confidence: 99%

Automated Cyber Defence: A Review

Vyas¹,

Hannay²,

Bolton³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent work from Milani et al [27] summarizes different methods of explanations in RL algorithms under a new taxonomy based on three main groups:…”

Section: Explainable Reinforcement Learningmentioning

confidence: 99%

Analysis of Explainable Goal-Driven Reinforcement Learning in a Continuous Simulated Environment

et al. 2022

View full text Add to dashboard Cite

Currently, artificial intelligence is in an important period of growth. Due to the technology boom, it is now possible to solve problems that could not be resolved previously. For example, through goal-driven learning, it is possible that intelligent machines or agents may be able to perform tasks without human intervention. However, this also leads to the problem of understanding the agent’s decision making. Therefore, explainable goal-driven learning attempts to eliminate this gap. This work focuses on the adaptability of two explainability methods in continuous environments. The methods based on learning and introspection proposed a probability value for success to explain the agent’s behavior. These had already been tested in discrete environments. The continuous environment used in this study is the car-racing problem. This is a simulated car racing game that forms part of the Python Open AI Gym Library. The agents in this environment were trained with the Deep Q-Network algorithm, and in parallel the explainability methods were implemented. This research included a proposal for carrying out the adaptation and implementation of these methods in continuous states. The adaptation of the learning method produced major changes, implemented through an artificial neural network. The obtained probabilities of both methods were consistent throughout the experiments. The probability result was greater in the learning method. In terms of computational resources, the introspection method was slightly better than its counterpart.

show abstract

“…Most work on interpretable RL focuses on the single-agent setting [23]. We first discuss techniques that directly learn DT policies.…”

Section: Related Workmentioning

confidence: 99%

MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent Reinforcement Learning

Milani¹,

Zhang²,

Topin³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Many recent breakthroughs in multi-agent reinforcement learning (MARL) require the use of deep neural networks, which are challenging for human experts to interpret and understand. On the other hand, existing work on interpretable RL has shown promise in extracting more interpretable decision tree-based policies, but only in the single-agent setting. To fill this gap, we propose the first set of algorithms that extract interpretable decision-tree policies from neural networks trained with MARL. The first algorithm, IVIPER, extends VIPER, a recent method for single-agent interpretable RL, to the multi-agent setting. We demonstrate that IVIPER can learn high-quality decision-tree policies for each agent. To better capture coordination between agents, we propose a novel centralized decision-tree training algorithm, MAVIPER. MAVIPER jointly grows the trees of each agent by predicting the behavior of the other agents using their anticipated trees, and uses resampling to focus on states that are critical for its interactions with other agents. We show that both algorithms generally outperform the baselines and that MAVIPER-trained agents achieve better-coordinated performance than IVIPER-trained agents on three different multi-agent particle-world environments.

show abstract

A Survey of Explainable Reinforcement Learning

Cited by 17 publications

References 6 publications

Automated Cyber Defence: A Review

Automated Cyber Defence: A Review

Analysis of Explainable Goal-Driven Reinforcement Learning in a Continuous Simulated Environment

MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent Reinforcement Learning

Contact Info

Product

Resources

About