Deep Reinforcement Learning: A New Beacon for Intelligent Active Flow Control

The optimal control of flow and fluid–structure interaction (FSI) systems often requires an accurate model of the controlled system. However, for strongly nonlinear systems, acquiring an accurate dynamic model is a significant challenge. In this study, we employ the deep reinforcement learning (DRL) method, which does not rely on an accurate model of the controlled system, to address the control of transonic buffet (unstable flow) and transonic buffeting (structural vibration). DRL uses a deep neural network to describe the control law and optimizes it based on data obtained from interaction between control law and flow or FSI system. This study analyzes the mechanism of transonic buffet and transonic buffeting to guide the design of control system. Aiming at the control of transonic buffet, which is an unstable flow system, the control law optimized by DRL can quickly suppress fluctuating load of buffet by taking the lift coefficient as feedback signal. For the frequency lock-in phenomenon in transonic buffeting flow, which is an unstable FSI system, we add the moment coefficient and pitching displacement to feedback signal to observe pitching vibration mode. The control law optimized by DRL can also effectively eliminate or reduce pitching vibration displacement of airfoil and buffet load. The simulation results in this study show that DRL can adapt to the control of two different dynamic modes: typical forced response and FSI instability under transonic buffet, so it has a wide application prospect in the design of control laws for complex flow or FSI systems.

Adaptive control of transonic buffet and buffeting flow with deep reinforcement learning

Ren,

Gao,

Xiong

et al. 2024

Robust and adaptive deep reinforcement learning for enhancing flow control around a square cylinder with varying Reynolds numbers

Jia,

2024

The present study applies a Deep Reinforcement Learning (DRL) algorithm to Active Flow Control (AFC) of a two-dimensional flow around a confined square cylinder. Specifically, the Soft Actor-Critic (SAC) algorithm is employed to modulate the flow of a pair of synthetic jets placed on the upper and lower surfaces of the confined squared cylinder in flow configurations characterized by Re of 100, 200, 300, and 400. The investigation starts with an analysis of the baseline flow in the absence of active control. It is observed that at Re = 100 and Re = 200, the vortex shedding exhibits mono-frequency characteristics. Conversely, at Re = 300 and Re = 400, the vortex shedding is dominated by multiple frequencies, which is indicative of more complex flow features. With the application of the SAC algorithm, we demonstrate the capability of DRL-based control in effectively suppressing vortex shedding, while significantly diminishing drag and fluctuations in lift. Quantitatively, the data-driven active control strategy results in a drag reduction of approximately 14.4%, 26.4%, 38.9%, and 47.0% for Re = 100, 200, 300, and 400, respectively. To understand the underlying control mechanism, we also present detailed flow field comparisons, which showcase the adaptability of DRL in devising distinct control strategies tailored to the dynamic conditions at varying Re. These findings substantiate the ability of DRL to control chaotic, multi-frequency dominated vortex shedding phenomena, underscoring the robustness of DRL in complex AFC problems.

Deep reinforcement learning-based active flow control of an elliptical cylinder: Transitioning from an elliptical cylinder to a circular cylinder and a flat plate

Jia,

2024

We study the adaptability of deep reinforcement learning (DRL)-based active flow control (AFC) technology for bluff body flows with complex geometries. It is extended from a cylinder with an aspect ratio Ar = 1 to a flat elliptical cylinder with Ar = 2, slender elliptical cylinders with Ar less than 1, and a flat plate with Ar = 0. We utilize the Proximal Policy Optimization (PPO) algorithm to precisely control the mass flow rates of synthetic jets located on the upper and lower surfaces of a cylinder to achieve reduction in drag, minimization of lift, and suppression of vortex shedding. Our research findings indicate that, for elliptical cylinders with Ar between 1.75 and 0.75, the reduction in drag coefficient ranges from 0.9% to 15.7%, and the reduction in lift coefficient ranges from 95.2% to 99.7%. The DRL-based control strategy not only significantly reduces lift and drag, but also completely suppresses vortex shedding while using less than 1% of external excitation energy, demonstrating its efficiency and energy-saving capabilities. Additionally, for Ar from 0.5 to 0, the reduction in drag coefficient ranges from 26.9% to 43.6%, and the reduction in lift coefficient from 50.2% to 68.0%. This reflects the control strategy's significant reduction in both drag and lift coefficients, while also alleviating vortex shedding. The interaction and nonlinear development of vortices in the wake of elliptical cylinders lead to complex flow instability, and DRL-based AFC technology shows adaptability and potential in addressing flow control problems for this type of bluff body flow.