Exploration of the applicability of probabilistic inference for learning control in underactuated autonomous underwater vehicles

Abstract

Underwater vehicles are employed in the exploration of dynamic environments where tuning of a specific controller for each task would be time-consuming and unreliable as the controller depends on calculated mathematical coefficients in idealised conditions. For such a case, learning task from experience can be a useful alternative. This paper explores the capability of probabilistic inference learning to control autonomous underwater vehicles that can be used for different tasks without re-programming the controller. Probabilistic inference learning uses a Gaussian process model of the real vehicle to learn the correct policy with a small number of real field experiments. The use of probabilistic reinforced learning looks for a simple implementation of controllers without the burden of coefficients calculation, controller tuning or system identification. A series of computational simulations were employed to test the applicability of model-based reinforced learning in underwater vehicles. Three simulation scenarios were evaluated: waypoint tracking, depth control and 3D path tracking control. The 3D path tracking is done by coupling together a line-of-sight law with probabilistic inference for learning control. As a comparison study LOS-PILCO algorithm can perform better than a robust LOS-PID. The results shows that probabilistic model based reinforced learning is a possible solution to motion control of underactuated AUVs as can generate capable policies with minimum quantity of episodes.

AUV Obstacle Avoidance Framework Based on Event-Triggered Reinforcement Learning

“…Jiang et al [21] also used the DDPG algorithm to control three degrees of freedom of the underwater vehicle and realized the uniform linear motion of the underwater vehicle. Other scholars have realized the control of 5-DOF AUV by improving deep RL [22]. Some researchers used the DQN algorithm and PPO algorithm to realize collision avoidance and multi-position tracking of AUV [23].…”

The Optimization of RBFNN Gearshift Controller Parameters for Electric Vehicles Using PILCO Reinforcement Learning

“…This approach enables precise and efficient control optimization with only a few experiments [22], [23]. Previous studies have applied PILCO to different engineering disciplines, including an autopilot underwater vehicle control problem [24], [25], constant force control of a robot surface [26], autonomous optimization of PID parameters in the control of a flight attitude simulator [27], and experiments on a planetary gear transmission shift console frame [28]. These studies have shown that PILCO can obtain the ideal controller strategy for the gearbox with a relatively small number of experiments.…”