A reinforcement learning approach to airfoil shape optimization

Abstract

Shape optimization is an indispensable step in any aerodynamic design. However, the inherent complexity and non-linearity associated with fluid mechanics as well as the high-dimensional design space intrinsic to such problems make airfoil shape optimization a challenging task. Current approaches relying on gradient-based or gradient-free optimizers are data-inefficient in that they do not leverage accumulated knowledge, and are computationally expensive when integrating Computational Fluid Dynamics (CFD) simulation tools. Supervised learning approaches have addressed these limitations but are constrained by user-provided data. Reinforcement learning (RL) provides a data-driven approach bearing generative capabilities. We formulate the airfoil design as a Markov decision process (MDP) and investigate a Deep Reinforcement Learning (DRL) approach to airfoil shape optimization. A custom RL environment is developed allowing the agent to successively modify the shape of an initially provided 2D airfoil and to observe the associated changes in aerodynamic metrics such as lift-to-drag (L/D), lift coefficient (Cl) and drag coefficient (Cd). The learning abilities of the DRL agent are demonstrated through various experiments in which the agent’s objective-maximizing L/D, maximizing Cl or minimizing Cd-as well as the initial airfoil shape are varied. Results show that the DRL agent is able to generate high performing airfoils within a limited number of learning iterations. The strong resemblance between the artificially produced shapes and those found in the literature highlights the rationality of the decision-making policy learned by the agent. Overall, the presented approach demonstrates the relevance of DRL to airfoil shape optimization and brings forward a successful application of DRL to a physics-based aerodynamics problem.

Actively learning costly reward functions for reinforcement learning

“…Another source of interest is the optimization of airfoils to improve their aerodynamic properties, with all kinds of applications in aeronautics. While traditionally these kinds of problems are tackled with optimization methods such as gradient-based optimization, the authors in [23] argue that these methods, even though computationally efficient in large spaces, are susceptible to poor local minima, and do not work well with non-linear cost functions. While machine learning techniques are less susceptible to these kinds of errors, the authors in [10] point out that using high-fidelity data for training can become prohibitively expensive.…”

A Deep Reinforcement Learning Approach to Optimal Morphologies Generation in Reconfigurable Tiling Robots

“…Shape Optimization Using Reinforcement Learning: In recent years, reinforcement learning (RL) has been extensively used to address the shape optimization problem. For example, in [42], the authors use deep RL for airfoil shape optimization. The RL agent modifies the shape of the 2D airfoil within geometric constraints, and it is run on a lowfidelity external solver to calculate the reward.…”

Deep Learning Models for the Evaluation of the Aerodynamic and Thermal Performance of Three-Dimensional Symmetric Wavy Wings

“…Wavy geometry is a biomimetic technique inspired from the unique structure of the fins of humpback whales. This geometry has been found to offer significant advantages in fluid mechanics and flow control, particularly in airfoil design [1][2][3][4][5][6][7][8][9][10][11][12][13]. By incorporating the wavy shape into airfoils, researchers have been able to achieve improvements in lift, drag, and overall aerodynamic performance.…”