A fuzzy Actor–Critic reinforcement learning network

Wang, Xuesong; Cheng, Yuhu; Yi, Jianqiang

doi:10.1016/j.ins.2007.03.012

Cited by 61 publications

(34 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Fuzzy net has an RBF-like architecture and powerful ability to classify continuous input and give continuous output (Jouffe, 1998). It has been successfully adopted in cart-pole balance control (Wang, X. S., et al, 2007), adaptive behavior learning of autonomous robots (Samejima and Omori, 1999), (Perez- Fig.1. Structure of a neuro-fuzzy learning system for adaptive swarm behaviors is shown.…”

Section: Fuzzy Netmentioning

confidence: 99%

“…We proposed a self-organized fuzzy neural network (SOFNN) which constructed adaptive membership functions and rules using training data and thresholds previously (Kuremoto, et al, 2003(Kuremoto, et al, , 2007(Kuremoto, et al, , 2008a, . Wang, Cheng, and Yi proposed a structure learning algorithm for adding and merging units using TDerror distribution recently (Wang, X. S., et al, 2007). Here we use a simpler selfmultiplication algorithm to decide the size of Fuzzy net.…”

Section: Fuzzy Netmentioning

confidence: 99%

“…However, a serious problem exists when RL is used in multi-agent systems is that the explosion of state space (curse of dimension). To overcome this hurdle, approaches using fuzzy inference systems (Jouffe, 1998), (Wang, X. S., et al, 2007), adaptive global radial basis function (GRBF) network (Samejima and Omori, 1999) are proposed, and recently classifying states using ART (Kobayashi et al, 2005), Neuro-GAS-like technique or hierarchical structure model (Wang, B. N. et al, 2007) are also given effectively. In this study, individuals are assumed without any knowledge of the environment, and they act according to a stochastic policy of the internal model temporally.…”

Section: An Actor-critic Learning Neworkmentioning

confidence: 99%

See 2 more Smart Citations

Adaptive swarm behavior acquisition by a neuro‐fuzzy system and reinforcement learning algorithm

Kuremoto

Obayashi

Kobayashi

2009

International Journal of Intelligent Computing and Cybernetics

View full text Add to dashboard Cite

Purpose -A neuro-fuzzy system with a reinforcement learning algorithm (RL) for adaptive swarm behaviors acquisition is presented. The basic idea is that each individual (agent) has the same internal model and the same learning procedure, and the adaptive behaviors are acquired only by the reward or punishment from the environment. The formation of the swarm is also designed by RL, e.g., TD-error learning algorithm, and it may bring out a faster exploration procedure comparing with the case of individual learning. Design/Methodology/Approach -The internal model of each individual composes a part of input states classification by a fuzzy net, and a part of optimal behavior learning network which adopting a kind of reinforcement learning methodology named actor-critic method. The membership functions and fuzzy rules in the fuzzy net are adaptively formed online by the change of environment states observed in the trials of agent's behaviors. The weights of connections between the fuzzy net and the action-value functions of Actor which provides a stochastic policy of action selection, and Critic which provides an evaluation to state transmission, are modified by temporal difference error (TD-error). Findings -Simulation experiments of the proposed system with several goal-directed navigation problems were accomplished and the results showed that swarms were successfully formed and optimized routes were found by swarm learning faster than the case of individual learning. Originality/value -Two techniques i.e. fuzzy identification system and reinforcement learning algorithm are fused into an internal model of the individuals for swarm formation and adaptive behavior acquisition. The proposed model may be applied to multi-agent systems, swarm robotics, metaheuristic optimization and so on.

show abstract

Section: Fuzzy Netmentioning

confidence: 99%

Section: Fuzzy Netmentioning

confidence: 99%

Section: An Actor-critic Learning Neworkmentioning

confidence: 99%

See 1 more Smart Citation

Adaptive swarm behavior acquisition by a neuro‐fuzzy system and reinforcement learning algorithm

Kuremoto

Obayashi

Kobayashi

2009

International Journal of Intelligent Computing and Cybernetics

View full text Add to dashboard Cite

show abstract

“…To tackle the first issue, linear approximation method [2], normalized Gaussian radial basis function classification method [3], fuzzy inference systems [4] - [8], etc. are proposed.…”

mentioning

confidence: 99%

“…Even the exploration environment is stable, the existence of other agents nearby the learner agent is uncertain. In [4] - [8], Kuremoto et al proposed to calculate the reward of suitable distance between agents to modify the action policy and showed its higher learning convergence comparing with RLs by individuals independently.…”

mentioning

confidence: 99%

How an Adaptive Learning Rate Benefits Neuro-Fuzzy Reinforcement Learning Systems

Kuremoto

Obayashi

Kobayashi

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract.To acquire adaptive behaviors of multiple agents in the unknown environment, several neuro-fuzzy reinforcement learning systems (NFRLSs) have been proposed Kuremoto et al. Meanwhile, to manage the balance between exploration and exploitation in fuzzy reinforcement learning (FRL), an adaptive learning rate (ALR), which adjusting learning rate by considering "fuzzy visit value" of the current state, was proposed by Derhami et al. recently. In this paper, we intend to show how the ALR accelerates some NFRLSs which are reinforcement learning systems with a self-organizing fuzzy neural network (SOFNN) and different learning methods including actor-critic learning (ACL), and Sarsa learning (SL). Simulation results of goal-exploration problems showed the powerful effect of the ALR comparing with the conventional empirical fixed learning rates.

show abstract

Q(λ)-learning adaptive fuzzy logic controllers for pursuit-evasion differential games

Desouky

Schwartz

2011

Int. J. Adapt. Control Signal Process.

View full text Add to dashboard Cite

This paper addresses the problem of tuning the input and the output parameters of a fuzzy logic controller. A novel technique that combines Q( )-learning with function approximation (fuzzy inference system) is proposed. The system learns autonomously without supervision or a priori training data. The proposed technique is applied to three different pursuit-evasion differential games. The proposed technique is compared with the classical control strategy, Q( )-learning only, and the technique proposed by Dai et al. (2005) in which a neural network is used as a function approximation for Q-learning. Computer simulations show the usefulness of the proposed technique.

show abstract

A fuzzy Actor–Critic reinforcement learning network

Cited by 61 publications

References 30 publications

Adaptive swarm behavior acquisition by a neuro‐fuzzy system and reinforcement learning algorithm

Adaptive swarm behavior acquisition by a neuro‐fuzzy system and reinforcement learning algorithm

How an Adaptive Learning Rate Benefits Neuro-Fuzzy Reinforcement Learning Systems

Q(λ)-learning adaptive fuzzy logic controllers for pursuit-evasion differential games

Contact Info

Product

Resources

About