Deep reinforcement learning for propulsive performance of a flapping foil

Abstract

While it is challenging for a traditional propulsor to achieve a wide range of force profile manipulation and propulsion efficiency, nature provides a solution for a flapping foil such as that found in birds and turtles. In this paper, we introduce a deep reinforcement learning (DRL) algorithm with great potential for solving nonlinear systems during the simulation to achieve a self-learning posture adjustment for a flapping foil to effectively improve its thrust performance. With DRL, a brute-force search is first carried out to provide intuition about the optimal trajectories of the foil and also a database for the following case studies. We implement an episodic training strategy for intelligent agent learning using the DRL algorithm. To address a slow data generation issue in the computational fluid dynamics simulation, we introduce a multi-environment technique to accelerate data exchange between the environment and the agent. This method is capable of adaptively and automatically performing an optimal foil path planning to generate the maximum thrust under various scenarios and can even outperform the optimal cases designed by users. Numerical results demonstrate how the proposed DRL is powerful to achieve optimization and has great potential to solve a more complex problem in the field of fluid mechanics beyond human predictability.