Adaptive swarm behavior acquisition by a neuro‐fuzzy system and reinforcement learning algorithm

Abstract

Purpose -A neuro-fuzzy system with a reinforcement learning algorithm (RL) for adaptive swarm behaviors acquisition is presented. The basic idea is that each individual (agent) has the same internal model and the same learning procedure, and the adaptive behaviors are acquired only by the reward or punishment from the environment. The formation of the swarm is also designed by RL, e.g., TD-error learning algorithm, and it may bring out a faster exploration procedure comparing with the case of individual learning. Design/Methodology/Approach -The internal model of each individual composes a part of input states classification by a fuzzy net, and a part of optimal behavior learning network which adopting a kind of reinforcement learning methodology named actor-critic method. The membership functions and fuzzy rules in the fuzzy net are adaptively formed online by the change of environment states observed in the trials of agent's behaviors. The weights of connections between the fuzzy net and the action-value functions of Actor which provides a stochastic policy of action selection, and Critic which provides an evaluation to state transmission, are modified by temporal difference error (TD-error). Findings -Simulation experiments of the proposed system with several goal-directed navigation problems were accomplished and the results showed that swarms were successfully formed and optimized routes were found by swarm learning faster than the case of individual learning. Originality/value -Two techniques i.e. fuzzy identification system and reinforcement learning algorithm are fused into an internal model of the individuals for swarm formation and adaptive behavior acquisition. The proposed model may be applied to multi-agent systems, swarm robotics, metaheuristic optimization and so on.

An Improved Internal Model of Autonomous Robots by a Psychological Approach

“…In [3], cooperative behavior acquisition of mobile robots is realized with reinforcement learning which is inspired by behaviorist psychology. We also proposed a neuro-fuzzy reinforcement learning system for swarm formation and adaptive swarm behavior acquisition of the swarm robots in the previous work [4][5][6][7]. Meanwhile, Ide and Nozawa group proposed an internal model in [8,9], which drives autonomous robots avoiding obstacles and exploring a goal in the unknown environments using a psychological model proposed by Russell in [10,11].…”

How an Adaptive Learning Rate Benefits Neuro-Fuzzy Reinforcement Learning Systems

“…To deal with continuous state space, an actor-critic type neuro-fuzzy reinforcement learning system is proposed by Kuremoto et al [5] [6]. The system is able to be used as an internal model of an autonomous agent which output a series of adaptive actions in the exploration of the unknown environment.…”

“…Actor-critic type neuro-fuzzy reinforcement learning system described in Section 2.2 was used as a conventional system [5] [6]. When adaptive learning rate (ALR) shown in Section 2.3 was adopted in, the change of the learning performance was investigated.…”

“…To determine the number of membership functions and rules of fuzzy net, a selforganized fuzzy neural network (SOFNN) which is constructed adaptive membership functions and rules driven by training data and thresholds automatically [5] [6]. The self-organizing process of SOFNN is given as follows.…”

“…are proposed. Specially, neuro-fuzzy systems with a data-driven type self-organizing fuzzy neural network (SOFNN) proposed in our previous works showed their adaptive state identification ability for different RL algorithms such as actor-critic learning [5] [6], Q-learning, and Sarsa learning [7].…”