Real-time implementation of deep reinforcement learning controller for speed tracking of robotic fish through data-assisted modeling

Abstract

This article proposes real-time speed tracking of two-link surface swimming robotic fish using a deep reinforcement learning (DRL) controller. Hydrodynamic modelling of robotic fish is done by virtue of Newtonian dynamics and Lighthill’s kinematic model. However, this includes external unsteady reactive forces that cannot be modeled accurately due to the distributed nature of hydrodynamic behavior. Therefore, a novel data-assisted dynamic model and control method is proposed for the speed tracking of robotic fish. Initially, the cruise speed motion data are collected through experiments. The water-resistance coefficient is estimated using the least mean square fit, which is then adopted in the model. Subsequently, a closed-loop discrete-time DRL controller trained through a soft actor-critic (SAC) agent is implemented through simulations. SAC overcomes the brittleness problem encountered by other policy gradient approaches by encouraging the policy network for maximum exploration and not assigning a higher probability to any single part of actions. Due to this robustness in the policy learning, the convergence error becomes low in RL-SAC than RL-DDPG controller. The simulation results verify that the DRL-SAC control with data-assisted modelling substantially improves the speed tracking performance. Further, this controller is validated in real-time, and it is observed that the SAC-trained controller tracks the desired speed more accurately than the DDPG controller.