A Behavior-Based Scheme Using Reinforcement Learning for Autonomous Underwater Vehicles

Abstract

Abstract-This paper presents a hybrid behavior-based scheme using reinforcement learning for high-level control of autonomous underwater vehicles (AUVs). Two main features of the presented approach are hybrid behavior coordination and semi on-line neural-Q_learning (SONQL). Hybrid behavior coordination takes advantages of robustness and modularity in the competitive approach as well as efficient trajectories in the cooperative approach. SONQL, a new continuous approach of the Q_learning algorithm with a multilayer neural network is used to learn behavior state/action mapping online. Experimental results show the feasibility of the presented approach for AUVs.Index Terms-Autonomous underwater vehicle (AUV), behavior-based control, neural networks, reinforcement learning.

Command filter based globally stable adaptive neural control for cooperative path following of multiple underactuated autonomous underwater vehicles with partial knowledge of the reference speed

“…Many general adaptive neural controllers ensure the SGUUB tracking stability for marine vehicles [1,[13][14][15][27][28][29] , provided that the trajectory stays within the neural active region throughout the control process. However, under the influence of external disturbances or an improper initialization, the tracking performance will be destroyed or even cause the instability of the system.…”

“…It is shown that, in the absence of unknown disturbances and modeling errors, the NN controller guarantees semiglobally uniformly ultimately bounded (SGUUB). In [29] , a semi on-line NN Q-learning algorithm is proposed to learn the behavior state/action mapping. Besides, the effectiveness of the presented control algorithm for AUV is shown by experimental results.…”

Intelligent Navigation for a Solar Powered Unmanned Underwater Vehicle

“…There are a lot of studies on trajectory generation for robots using various approaches to the problem (e.g., [5,[19][20][21][22][23]). Some of the previous models (e.g., [21][22]24]) use global methods to search for possible paths in the workspace, which normally deal with static environments only and are computationally expensive when the environment is complex. Seshadri and Ghosh [1] proposed a new path-planning model using an iterative approach.…”

Mission specification in underwater robotics

“…In Fig. 9 some common mission specification designs are depicted: (a) Petri nets, used in ProCoSa [1] and CORAL [19], [20], [7] systems, (b) tasks, used in ITOCA [21] and others [22], and (c) behaviors [4], [22]. Their main features are summarized below: 1) Petri Nets: Petri Net formalism brings robustness and reliability.…”

“…The specification with Petri nets comes with software tools, but they have to be adapted to AUV mission specification. It is also common to find robotic systems that manage behaviors and learning algorithms [4], but their mission specification power is weaker.…”