Interpretable policies for reinforcement learning by empirical fuzzy sets

Huang, Jianfeng; Angelov, Plamen; Yin, Chengliang

doi:10.1016/j.engappai.2020.103559

Cited by 10 publications

(5 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a typical reinforcement learning framework, an agent learns to achieve a goal by interacting with the environment, which is defined in the form of a Markov decision process. The agent gets either rewards or penalties for the actions it performs, and its main goal is to maximize the long-term reward (Huang et al 2020).…”

Section: Reinforcement Learning-based Fuzzy Systemsmentioning

confidence: 99%

“…By treating the combined return value of a series of actions as the fitness value to be maximized, a particle swarm reinforcement learning method is presented in Hein et al (2017) to learn the best policy represented by fuzzy rules. Since the majority of existing fuzzy reinforcement learning methods are implemented on the basis of (fuzzy) neural networks with very limited interpretability, an interpretable reinforcement learning scheme is proposed in Huang et al (2020), where the learned policy can be expressed as human-intelligible IF-THEN rules and the value function is approximated through the AnYa type fuzzy rule-based system.…”

Section: Reinforcement Learning-based Fuzzy Systemsmentioning

confidence: 99%

See 1 more Smart Citation

Autonomous learning for fuzzy systems: a review

Han

Shen

et al. 2022

Artif Intell Rev

Self Cite

View full text Add to dashboard Cite

As one of the three pillars in computational intelligence, fuzzy systems are a powerful mathematical tool widely used for modelling nonlinear problems with uncertainties. Fuzzy systems take the form of linguistic IF-THEN fuzzy rules that are easy to understand for human. In this sense, fuzzy inference mechanisms have been developed to mimic human reasoning and decision-making. From a data analytic perspective, fuzzy systems provide an effective solution to build precise predictive models from imprecise data with great transparency and interpretability, thus facilitating a wide range of real-world applications. This paper presents a systematic review of modern methods for autonomously learning fuzzy systems from data, with an emphasis on the structure and parameter learning schemes of mainstream evolving, evolutionary, reinforcement learning-based fuzzy systems. The main purpose of this paper is to introduce the underlying concepts, underpinning methodologies, as well as outstanding performances of the state-of-the-art methods. It serves as a one-stop guide for readers learning the representative methodologies and foundations of fuzzy systems or who desire to apply fuzzy-based autonomous learning in other scientific disciplines and applied fields.

show abstract

Section: Reinforcement Learning-based Fuzzy Systemsmentioning

confidence: 99%

Section: Reinforcement Learning-based Fuzzy Systemsmentioning

confidence: 99%

Autonomous learning for fuzzy systems: a review

Han

Shen

et al. 2022

Artif Intell Rev

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is a type of decision maker for handling vague inputs. It has a capability of decision making similar to human that is by framing set of rules [21]. Several inputs are considered at an interval and appropriate action is obtained [22].…”

Section: Modelmentioning

confidence: 99%

Multi-Agent System for Resource Adaptation in IoT to Monitor Leakage in Irrigation Tanks

Hatti

Sutagundar²

2021

Preprint

View full text Add to dashboard Cite

The sporadic changes in the requirements by the end users has led to the problems in managing the resources of IoT devices. The problem of managing the heterogeneous requests with the available resources in view of ensuring Quality of service (QoS) to end users is challenging issue. The proposed model inculcates the adaptation policy for adapting the resources to fulfill the requirements of the user. The dynamic changes in the environment are handled by Reinforcement learning model with Fuzzy Interference system to apply the policy. Periodically monitoring of irrigation tank to alert the flow of water above Full Tank Level (FTL) by Reinforcement learning agent, prioritization of tasks (requests) by Fuzzy logic is performed. It is done by interacting through agency, providing video conferencing or video calling facility to the user based on availability of the user resources. It also adapts according to communication and computational resources. The proposed model is simulated to monitor and control the leakage in tank. It helps to remotely control the leakages in irrigation tanks/ bridges through Multi-Agent Fuzzy Q learning model. It focuses on adapting the resources of the devices and the action considering the user resources. The algorithm is simulated in Ifogsim and python and performance are evaluated in terms of resource cost, latency, execution time, energy consumption and network usage.

show abstract

“…As, for instance, knowing that two actions have nearly the same probability in a given state indicates that they are both equally good, which allows our rule mining algorithm to chose one action or the other, depending on what leads to the simplest rule set. This exploitation of the meta-information of an RL process also differentiates our work from approaches that translate the RL policy in a set of fuzzy rules, such as [11] and [9].…”

Section: Related Workmentioning

confidence: 99%

“…0 0 0 0 0 0 9 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0 0 9 0 9 9 0 0 0 23 0 11 11 As explained above, we propose a two-phased distillation algorithm, to produce meaningful rules from a black-box policy learned with Deep Reinforcement Learning. The first phase produces a list of rules that approximate how the Deep RL policy maps states to actions.…”

Section: Explaining Policies Through Distillationmentioning

confidence: 99%

Synthesising Reinforcement Learning Policies through Set-Valued Inductive Rule Learning

Coppens,

Steckelmacher,

Jonker

et al. 2021

Preprint

View full text Add to dashboard Cite

Today's advanced Reinforcement Learning algorithms produce black-box policies, that are often difficult to interpret and trust for a person. We introduce a policy distilling algorithm, building on the CN2 rule mining algorithm, that distills the policy into a rule-based decision system. At the core of our approach is the fact that an RL process does not just learn a policy, a mapping from states to actions, but also produces extra meta-information, such as action values indicating the quality of alternative actions. This meta-information can indicate whether more than one action is near-optimal for a certain state. We extend CN2 to make it able to leverage knowledge about equally-good actions to distill the policy into fewer rules, increasing its interpretability by a person. Then, to ensure that the rules explain a valid, non-degenerate policy, we introduce a refinement algorithm that fine-tunes the rules to obtain good performance when executed in the environment. We demonstrate the applicability of our algorithm on the Mario AI benchmark, a complex task that requires modern reinforcement learning algorithms including neural networks. The explanations we produce capture the learned policy in only a few rules, that allow a person to understand what the black-box agent learned. Source code: https://gitlab.ai.vub.ac.be/yocoppen/svcn2.

show abstract

Interpretable policies for reinforcement learning by empirical fuzzy sets

Cited by 10 publications

References 29 publications

Autonomous learning for fuzzy systems: a review

Autonomous learning for fuzzy systems: a review

Multi-Agent System for Resource Adaptation in IoT to Monitor Leakage in Irrigation Tanks

Synthesising Reinforcement Learning Policies through Set-Valued Inductive Rule Learning

Contact Info

Product

Resources

About