Explainable Reinforcement Learning: A Survey

Puiutta, Erika; Veith, Eric Msp

doi:10.48550/arxiv.2005.06247

Cited by 11 publications

(13 citation statements)

References 32 publications

(60 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some surveys focus on a particular type of technique for explainable ML. For instance, [58] and [59] survey web technologies and reinforcement learning-based approaches for explainability. Baum et al [60] on the other hand, provides a survey of AI projects on ethics, risk, and policy.…”

Section: Related Surveysmentioning

confidence: 99%

Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges

Ahmad¹,

Maabreh²,

Ghaly³

et al. 2020

Preprint

View full text Add to dashboard Cite

As we make tremendous advances in machine learning and artificial intelligence (ML/AI) technosciences, there is a renewed understanding in the ML/AI community that we must ensure that humans being are at the center of our deliberations so that we don't end in technology-induced dystopias. As strongly argued by Green in his book "Smart Enough City", the incorporation of technology in city environs does not automatically translate into prosperity, wellbeing, urban livability, or social justice. There is a great need to deliberate on the future of the cities worth living and designing. There are philosophical and ethical questions involved along with various challenges that relate to the security, safety, and interpretability of ML/AI algorithms that will form the technological bedrock of future cities. A number of research institutes on humancentered AI (HAI) have been established at top international universities-including Stanford Institute for Human-Centered Artificial Intelligence (HAI), Berkeley's Center for Human-Compatible AI (CHAI), and MIT Institute for Data, Systems, and Society (IDSS). Globally there are calls for technology to be made more humane and human-compatible. For example, Stuart Russell has a book called Human-Compatible AI. The Center for Humane Technology advocates for regulators and technology companies to avoid business models and product features that contribute to social problems such as extremism, polarization, misinformation, and Internet addiction. In this paper, we analyze and explore key challenges including security, robustness, interpretability, and ethical (data and algorithmic) challenges to a successful deployment of AI/ML in human-centric applications, with a particular emphasis on the convergence of these concepts/challenges. We provide a detailed review of existing literature on these key challenges and analyze how one of these challenges may lead to others or help in solving other challenges. The paper also advises on the current limitations, pitfalls, and future directions of research in these domains, and how it can fill the current gaps and lead to better solutions. We believe such rigorous analysis will provide a baseline for future research in the domain.

show abstract

Section: Related Surveysmentioning

confidence: 99%

Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges

Ahmad¹,

Maabreh²,

Ghaly³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…According to [62], the term interpretability is defined as the ability to not only explain the model's decisions but also to present these explanations in an understandable way to offer the possibility for non-expert users to predict the model's behavior. A common taxonomy classifies interpretability models along two main dimensions: the scope or level of the explanans (global vs local) and the time when the explanations are generated (intrinsic vs post-hoc).…”

Section: C) On Explainable Rl (Xrl)mentioning

confidence: 99%

“…Designing more understandable approaches for a broader audience is also a pressing issue to overcome in XRL. We refer the reader to [62], [63], and the references therein for a detailed overview of explainability methods in RL.…”

Section: C) On Explainable Rl (Xrl)mentioning

confidence: 99%

Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial

Feriani¹,

Hossain²

2020

Preprint

View full text Add to dashboard Cite

Deep Reinforcement Learning (DRL) has recently witnessed significant advances that have led to multiple successes in solving sequential decision-making problems in various domains, particularly in wireless communications. The future sixth-generation (6G) networks are expected to provide scalable, low-latency, ultra-reliable services empowered by the application of data-driven Artificial Intelligence (AI). The key enabling technologies of future 6G networks, such as intelligent metasurfaces, aerial networks, and AI at the edge, involve more than one agent which motivates the importance of multi-agent learning techniques. Furthermore, cooperation is central to establishing self-organizing, self-sustaining, and decentralized networks. In this context, this tutorial focuses on the role of DRL with an emphasis on deep Multi-Agent Reinforcement Learning (MARL) for AI-enabled 6G networks. The first part of this paper will present a clear overview of the mathematical frameworks for single-agent RL and MARL. The main idea of this work is to motivate the application of RL beyond the model-free perspective which was extensively adopted in recent years. Thus, we provide a selective description of RL algorithms such as Model-Based RL (MBRL) and cooperative MARL and we highlight their potential applications in 6G wireless networks. Finally, we overview the state-of-the-art of MARL in fields such as Mobile Edge Computing (MEC), Unmanned Aerial Vehicles (UAV) networks, and cell-free massive MIMO, and identify promising future research directions. We expect this tutorial to stimulate more research endeavors to build scalable and decentralized systems based on MARL.

show abstract

“…This problem becomes more severe in long-horizon tasks [3]. Another problem in deep RL is the lack of interpretability [28,34]. The learned behavior based on the black-box neural network is nontransparent and difficult to explain and understand.…”

Section: Introductionmentioning

confidence: 99%

Interpretable Model-based Hierarchical Reinforcement Learning using Inductive Logic Programming

Xu,

Fekri

2021

Preprint

View full text Add to dashboard Cite

Recently deep reinforcement learning has achieved tremendous success in wide ranges of applications. However, it notoriously lacks data-efficiency and interpretability. Data-efficiency is important as interacting with the environment is expensive. Further, interpretability can increase the transparency of the blackbox-style deep RL models and hence gain trust from the users. In this work, we propose a new hierarchical framework via symbolic RL, leveraging a symbolic transition model to improve the data-efficiency and introduce the interpretability for learned policy. This framework consists of a high-level agent, a subtask solver and a symbolic transition model. Without assuming any prior knowledge on the state transition, we adopt inductive logic programming (ILP) to learn the rules of symbolic state transitions, introducing interpretability and making the learned behavior understandable to users. In empirical experiments, we confirmed that the proposed framework offers approximately between 30% to 40% more data efficiency over previous methods.

show abstract

Explainable Reinforcement Learning: A Survey

Cited by 11 publications

References 32 publications

Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges

Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges

Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial

Interpretable Model-based Hierarchical Reinforcement Learning using Inductive Logic Programming

Contact Info

Product

Resources

About