Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning

Abstract

This paper focuses on the active flow control of a computational fluid dynamics simulation over a range of Reynolds numbers using deep reinforcement learning (DRL). More precisely, the proximal policy optimization (PPO) method is used to control the mass flow rate of four synthetic jets symmetrically located on the upper and lower sides of a cylinder immersed in a two-dimensional flow domain. The learning environment supports four flow configurations with Reynolds numbers 100, 200, 300, and 400, respectively. A new smoothing interpolation function is proposed to help the PPO algorithm learn to set continuous actions, which is of great importance to effectively suppress problematic jumps in lift and allow a better convergence for the training process. It is shown that the DRL controller is able to significantly reduce the lift and drag fluctuations and actively reduce the drag by ∼5.7%, 21.6%, 32.7%, and 38.7%, at Re = 100, 200, 300, and 400, respectively. More importantly, it can also effectively reduce drag for any previously unseen value of the Reynolds number between 60 and 400. This highlights the generalization ability of deep neural networks and is an important milestone toward the development of practical applications of DRL to active flow control.

Robust flow control and optimal sensor placement using deep reinforcement learning

“…Unlike Tang et al. (2020), who trained their policy on several Reynolds-number values (, , and ) and evaluated it on a mix of ‘seen’ and ‘unseen’ Reynolds numbers, our policy has been trained on a single Reynolds-number value , and evaluations have been performed on a range spanning from to and compared with cases specifically trained on those Reynolds numbers. As illustrated by figure 3, this range of Reynolds numbers corresponds to a variation in vortex shedding Strouhal number of around .…”

“…Tang et al. (2020) used the same non-dimensionalisation scheme. Thus, even though their deep learning algorithm is different, the robustness they observed may be explained by the fact that the policy is robust over a wide range of Reynolds numbers even with a single Reynolds-number training.…”

“…(2000) and Tang et al. (2020), who achieved and at , respectively, show that this is possible. However, in the case of He et al.…”

“…This issue has been explored by Tang et al. (2020) and Ren, Rabault & Tang (2020), who successfully controlled a 2-D cylinder wake across a large range of Reynolds numbers. These applications show the relevance of complex control over linear-approximation-based control for this kind of nonlinear system.…”

“…(2019) and Tang et al. (2020), respectively, use 151 and 236 probes to control the flow past a 2-D cylinder. Their work is therefore a first step for DRL control, which needs to be continued and further improved, which is precisely the purpose of this work.…”