Q(λ)-learning adaptive fuzzy logic controllers for pursuit-evasion differential games

Abstract

This paper addresses the problem of tuning the input and the output parameters of a fuzzy logic controller. A novel technique that combines Q( )-learning with function approximation (fuzzy inference system) is proposed. The system learns autonomously without supervision or a priori training data. The proposed technique is applied to three different pursuit-evasion differential games. The proposed technique is compared with the classical control strategy, Q( )-learning only, and the technique proposed by Dai et al. (2005) in which a neural network is used as a function approximation for Q-learning. Computer simulations show the usefulness of the proposed technique.

An Intelligent Control System for Mobile Robot Navigation Tasks in Surveillance

“…In [4], using velocity vectors of the robot relative to each obstacle, an online navigation method based on calculating the best feasible direction close to an optimal direction to the target is proposed for pursuing a moving target amidst dynamic and static obstacles. Adaptive learning control for pursuit-evasion were presented in [6], [7], and experiments on capturing a moving object using pure pursuit were shown in [8]. …”

“…Applications of robotics have been applied to home services, health care and military missions such [3]- [5], etc. Developing various intelligence services, for example intelligent surveillance and patrol systems, is of emerging demand to support human society [6]- [7]. As an intelligent mechatronics system, the mobile robot needs to integrate algorithms related to environment sensing for obstacle detection and SLAM, behavior and route planning, controlling and executing [8].…”

Heuristic dynamic programming using echo state network as online trainable adaptive critic

“…In each episode, we let the robot move forward and backward in front of the obstacle in order to acquire rewards and react according to the control policy (10). The episode finishes (corresponding to k D k f / when the robot enters the shaded zone (shown in Figure 4), that is, the distance threshold th was reached.…”

The residual gradient FACL algorithm for differential games

“…Different from supervised learning, which is learning from input-output data provided by an expert, reinforcement learning is adequate for learning from interaction by using very simple evaluative or critic information instead of instructive information [1]. Reinforcement learning has been used by some authors as a mechanism in tuning and adaptation of the fuzzy logic controllers [3], [7], [12], [13], [22]. Some of the commonly used reinforcement learning algorithms estimate the value function of the state-action pairs, where the estimated value function shows how good it is for the learning agent to perform a given action in a given state.…”

“…This is because the Q-learning algorithm can only deal with learning environments that have discrete states and actions. Different algorithms that extend the Q-learning method Mostafa to deal with differential games by using fuzzy inference systems have been proposed in literature [3], [7], [12], [13], [15], [22]. One of these algorithms is the Q-learning fuzzy inference system (QLFIS) algorithm proposed in [7].…”