Applying deep reinforcement learning to active flow control in weakly turbulent conditions

Abstract

Machine learning has recently become a promising technique in fluid mechanics, especially for active flow control (AFC) applications. A recent work [Rabault et al., J. Fluid Mech. 865, 281-302 (2019)] has demonstrated the feasibility and effectiveness of deep reinforcement learning (DRL) in performing AFC over a circular cylinder at Re ¼ 100, i.e., in the laminar flow regime. As a follow-up study, we investigate the same AFC problem at an intermediate Reynolds number, i.e., Re ¼ 1000, where the weak turbulence in the flow poses great challenges to the control. The results show that the DRL agent can still find effective control strategies, but requires much more episodes in the learning. A remarkable drag reduction of around 30% is achieved, which is accompanied by elongation of the recirculation bubble and reduction of turbulent fluctuations in the cylinder wake. Furthermore, we also perform a sensitivity analysis on the learnt control strategies to explore the optimal layout of sensor network. To our best knowledge, this study is the first successful application of DRL to AFC in weakly turbulent conditions. It therefore sets a new milestone in progressing toward AFC in strong turbulent flows.

Reinforcement-learning-based control of confined cylinder wakes with stability analyses

“…Paris, Beneddine & Dandois (2021) used a stochastic gated input layer in the RL agent to select an optimal subset from some initially placed probes. Ren, Rabault & Tang (2021) performed a follow-up study of and presented a successful application of the RL control in weakly turbulent conditions (Re = 1000) with a drag reduction of 30 %. Beintema et al (2020) applied RL in the suppression of Rayleigh-Bénard convection and discussed limitations in controlling unstable and chaotic dynamics.…”

Flow Control in Wings and Discovery of Novel Approaches via Deep Reinforcement Learning

“…Xu et al [124] investigated in a simulation how small counter-rotating cylinders can be used to reduce the drag behind a cylinder, while Fan et al [125] provided an experimental demonstration of the technique. Finally, Ren et al [126] pushed the value of the Reynolds number to a weakly turbulent regime and demonstrated that DRL can control fluid motion in the turbulent regime.…”

Hybrid analysis and modeling, eclecticism, and multifidelity computing toward digital twin revolution

“…The main framework of the RL consists of an agent (e.g., a neural network in deep RL) that interacts with an environment to learn a policy that will maximize the cumulative reward over a long time horizon [315]. In recent years, the RL has been explored for fluid dynamics problems including animal locomotion [116,279,339], control of chaotic dynamics [41,59,337], drag reduction of bluff bodies [271,282,330], flow separation control [307], and turbulence closure modeling [242]. Along with a computer simulation environment, RL has been effectively applied for active flow control around bluff bodies in an experimental setup [98].…”