CDT: Cascading Decision Trees for Explainable Reinforcement Learning

Abstract

Deep Reinforcement Learning (DRL) has recently achieved significant advances in various domains. However, explaining the policy of RL agents still remains an open problem due to several factors, one being the complexity of explaining neural networks decisions. Recently, a group of works have used decision-tree-based models to learn explainable policies. Soft decision trees (SDTs) and discretized differentiable decision trees (DDTs) have been demonstrated to achieve both good performance and share the benefit of having explainable policies. In this work, we further improve the results for tree-based explainable RL in both performance and explainability. Our proposal, Cascading Decision Trees (CDTs) apply representation learning on the decision path to allow richer expressivity. Empirical results show that in both situations, where CDTs are used as policy function approximators or as imitation learners to explain black-box policies, CDTs can achieve better performances with more succinct and explainable models than SDTs. As a second contribution our study reveals limitations of explaining black-box policies via imitation learning with tree-based explainable models, due to its inherent instability.

Exploring online and offline explainability in deep reinforcement learning for aircraft separation assurance

“…Univariate nodes are introduced to discretize SDTs for better interpretability (Silva et al, 2020). A feature learning tree is integrated with the standard SDT to improve model expressivity (Ding et al, 2020). A series of novel metrics are proposed for a comprehensive evaluation (Dahlin et al, 2020).…”

“…2) Our work concentrates on the aircraft separation assurance task with a complex high-dimensional input space. But the previous works mainly focus on tasks with simple low-dimensional input [e.g., CartPole (Ding et al, 2020), LunarLander (Silva et al, 2020)] or tasks with pixel-based input [e.g., Mario AI Benchmark (Coppens et al, 2019), Wildfire Tracking (Haksar and Schwager, 2018)]. 3) Our work focuses on a complex multi-agent problem so the proposed methods need to consider the cooperation among all agents in the system.…”